WIP: copy config function source into the compiled file

[wbthomason]

Just something I was playing around with - by adapting the functions given in the answer here: https://stackoverflow.com/questions/29871102/in-trepl-or-luajit-how-can-i-find-the-source-code-of-a-library-im-using/29872347#29872347 we can extract the source of a user's config (or eventually setup, etc.) functions and put them directly in the compiled output, rather than using loadstring.

This is kinda cool, but it's not clear that it's helpful - on the one hand, functions created with loadstring are never JIT'd, but on the other hand, these functions should only run once and are unlikely to benefit from JITing. This approach also creates code duplication by copying the function source to the compiled file.

I'm mostly just posting this draft PR in case others see it and can think of a reason that it's worth pursuing further. This PR is 90% working, but I need to fix the output of the packer_plugins table to capture a reference to the config functions rather than just the string of their names.

[akinsho]

@wbthomason does this mean that the compiled output would actually contain the functions themselves from users config e.g. the setup and config functions or do you mean the entire function that contains their use statements? so it would be a copy of a users say plugins.lua file(or part of it)?

[wbthomason]

@akinsho: Just the setup and config functions, not the entire function containing use statements.

[akinsho]

Well it's probably not a major point but not sure what the tradeoffs are, the duplication I think is unlikely to be particularly meaningful to most people andI think the ability to actually be able to inspect that file would be a huge win for readability 😄 . When I was working on the compile related reloading, trying to figure out what was happening meant looking for changes amongst the gibberish in the loadstring was a pain also I think anyone trying to figure out what's going on can more easily make sense of things 🤷🏿

[wbthomason]

True, legibility would be a benefit.

[akinsho]

On a separate but vaguely related note, I spent a bit of time looking into the upvalue stuff using debug info and getting values from scopes I wonder if you've run into anything about that.

This link shows some of the debug functions. One of the ones that was particularly interesting was debug.getupvalue not sure if that is a remotely feasible way of resolving or mitigating the upvalue gotcha

[kevinhwang91]

The reason I don't use config and setup is loadstring is slow. I prefer to write my config in lua file rather than in plugins.lua.

[akinsho]

@kevinhwang91 do you have any links to benchmarks or posts describing the slowness of loadstring, to be clear I'm just curious not challenging what you're saying, I just find stuff like that interesting.

[akinsho]

maybe just profiling the execution/startup time with this PR vs. the default behaviour would be enough to figure out if loadstring is slower.

[wbthomason]

My own ad hoc benchmarks of the two were inconclusive. Additionally, while loadstring from a string representing Lua syntax can be slow, intuitively it seems like loadstring of bytecode (as packer uses) shouldn't be much slower than parsing and running a function (aside from the aforementioned lack of JITing).

@kevinhwang91, if you have examples showing that loadstring is in fact slower for this case, I'd be very interested to see them.

[kevinhwang91]

@kevinhwang91 do you have any links to benchmarks or posts describing the slowness of loadstring, to be clear I'm just curious not challenging what you're saying, I just find stuff like that interesting.

I'm so sorry for my preconceived bias, I learn lua from https://www.lua.org/pil/8.html which said:

The loadstring function is powerful; it must be used with care. It is also an expensive function (when compared to its alternatives) and may result in incomprehensible code. Before you use it, make sure that there is no simpler way to solve the problem at hand.

I simply run lua local start = vim.fn.reltime() local f = loadstring("\27LJ\2\n@\0\0\1\0\0\1\14)\0\0\0\22\0\0\0\22\0\0\0\22\0 \0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0\22\0\0\0K\0\1\0\2\0") print(vim.fn.r eltimestr(vim.fn.reltime(start))), which explains loadstring costs very low compare with lua local start = vim.fn.reltime() print(vim.fn.reltimestr(vim.fn.reltime(start))).

the original function is:

            setup = function()
                local i = 0
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
                i = i + 1
            end

so luastring without any performance issue.