Cleaning up old LuaChunk instances

Aug 30, 2014 at 2:03 AM
Me again! :)

So I'm reloading a lot of scripts in new LuaGlobal environments at runtime (every level of my game server has a set of scripts, and levels change fairly often), however am keeping them in the same Lua instance (mainly so I can precompile a few 'system' scripts as chunks as otherwise loading becomes really slow).

However, for every reload, around 600 kB of managed heap data gets added to the Lua instance - even though I'm removing all my references to the LuaChunk and my old LuaGlobal, probably because the items stored in the old global state (and the chunk) are stored in whatever cache the Lua instance is keeping for dynamic method purposes seemingly.

It's not a high-priority issue right now, but I foresee a few annoyances in the future when my project ends up getting used a lot more than it does now :)
Coordinator
Aug 30, 2014 at 6:37 AM
No problem, your are welcome.

Do you compile the scripts with debug flag on?

If not the LuaChunk should get removed. And it removes all references to the code with a call on dispose. But I am not really sure.
What not will be removed is the dynamic cache, but this cache should not grow after a while.

For me it has a high priority, because I run with this library a 24/7 service. With the version I use, currently. I have no problems, may be I messed up something in the last development steps.

Do you have any profile results?
Aug 30, 2014 at 2:59 PM
Yes, the chunks do seem to get released - the profiler seems to show that some of the internal fields of the SetMemberBinder are growing in size every reload however - see a screenshot of such relevant parts of the VS2013 memory view between two snapshots.
Coordinator
Aug 30, 2014 at 9:20 PM
Hui,

two important questions:
Do you reload the same script?
Do you compile the scripts with debug flag on?

If you reload the same script the cache should not grow. In theory?

I will do some tests on this...
Aug 30, 2014 at 9:21 PM
Yes, this happens both when reloading the same script and when reloading (slightly, only differing by a small 4-line file) different scripts, and no, I'm not using the debugging flag.
Coordinator
Aug 30, 2014 at 10:01 PM
Edited Aug 30, 2014 at 10:07 PM
It seems that the Callsite's are growing in your application. The binders look, okay. Currently, I have no idea why.

In my little test all work fine, no growing of CallSite's at all?

Have you a chance to create a small example? May be it has todo something with the script? I want to know what is going on?

EDIT: Or can you send me the memory profile results?

FYI: If the debug flag is on, currently, the code will not collected, because I hold the callsite's and genereted types in the Lua-class :(.
Aug 30, 2014 at 10:34 PM
I used the following code to count the instances of rules cached in the [SetMemberBinder].Cache[(type of member)]._rules property (which seems to be holding the references to the Func<> with its RTDynamicMethod and Closure that are being kept alive):
            // where ms_luaState is the globally-kept Lua instance
            var field = ms_luaState.GetType().GetField("setMemberBinder", BindingFlags.NonPublic | BindingFlags.Instance);
            var binders = (Dictionary<string, System.Runtime.CompilerServices.CallSiteBinder>)field.GetValue(ms_luaState);

            Console.WriteLine("--- BOUNDARY ---");

            foreach (var binder in binders)
            {
                var fields = binder.Value.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Public | BindingFlags.Instance | BindingFlags.Static);

                if (fields.Length < 2)
                {
                    continue;
                }

                field = fields[1];
                var cache = (Dictionary<Type, Object>)field.GetValue(binder.Value);

                if (cache == null)
                {
                    continue;
                }

                foreach (var val in cache)
                {
                    field = val.Value.GetType().GetFields(BindingFlags.NonPublic | BindingFlags.Instance).FirstOrDefault(a => a.Name == "_rules");

                    if (field != null)
                    {
                        var rules = field.GetValue(val.Value);

                        var prop = rules.GetType().GetProperty("Length");
                        Console.WriteLine("{0}: {1}", binder.Key, prop.GetValue(rules));
                    }
                }
            }
and ended up with the following results:

Before
packers: 31
set_string: 31
set_array: 31
set_integer: 31
set_number: 31
pack: 86
__index: 117
types_map: 31
unpackers: 31
i: 128
build_ext: 62
s: 128
j: 128
underflow: 128
unpack: 86
unpacker: 31
_VERSION: 86
_DESCRIPTION: 31
_COPYRIGHT: 31
msgpack: 31
version: 41
getmetatable: 55
assert: 55
error: 55
ipairs: 55
next: 55
pairs: 55
pcall: 55
print: 55
rawequal: 55
rawget: 55
rawlen: 55
rawset: 55
select: 55
setmetatable: 55
tonumber: 55
tostring: 55
type: 55
xpcall: 55
arshift: 55
band: 55
bnot: 55
bor: 55
btest: 55
bxor: 55
extract: 55
lrotate: 55
lshift: 55
replace: 55
rrotate: 55
rshift: 55
bit32: 55
create: 55
resume: 55
running: 55
status: 55
wrap: 55
yield: 55
coroutine: 55
abs: 55
acos: 55
asin: 55
atan: 55
atan2: 55
ceil: 55
cos: 55
cosh: 55
deg: 55
exp: 55
floor: 55
fmod: 55
frexp: 55
huge: 55
ldexp: 55
log: 55
max: 55
min: 55
modf: 55
pi: 55
pow: 55
rad: 55
random: 55
randomseed: 55
sin: 55
sinh: 55
sqrt: 55
tan: 55
tanh: 55
math: 55
byte: 55
char: 55
dump: 55
find: 55
format: 55
gmatch: 55
gsub: 55
len: 55
lower: 55
match: 55
rep: 55
reverse: 55
sub: 55
upper: 55
string: 55
concat: 55
insert: 55
remove: 55
sort: 55
table: 55
server_scripts: 14
server_script: 14
description: 41
client_scripts: 41
client_script: 41
files: 41
file: 41
dependencies: 41
dependency: 41
__metatable: 2
CreateRPCContext: 31
maps: 1
gametypes: 1
getCurrentGameType: 1
getCurrentMap: 1
changeGameType: 1
changeMap: 1
doesMapSupportGameType: 1
name: 5
race: 3
gameTypes: 7
usjbattle: 1
lovelier: 1
play: 2
ignore_lod_modelinfos: 1
world_definition: 1
modelinfo_deadlock_hack: 1
bounds_arent_cdimage: 1
entity_sanity: 1
static_bound_sanity: 1
odd_wait_deadlock: 1
msgType: 1
hostname: 1
maxplayers: 1
After
packers: 33
set_string: 33
set_array: 33
set_integer: 33
set_number: 33
pack: 92
__index: 125
types_map: 33
unpackers: 33
i: 128
build_ext: 66
s: 128
j: 128
underflow: 128
unpack: 92
unpacker: 33
_VERSION: 92
_DESCRIPTION: 33
_COPYRIGHT: 33
msgpack: 33
version: 43
getmetatable: 59
assert: 59
error: 59
ipairs: 59
next: 59
pairs: 59
pcall: 59
print: 59
rawequal: 59
rawget: 59
rawlen: 59
rawset: 59
select: 59
setmetatable: 59
tonumber: 59
tostring: 59
type: 59
xpcall: 59
arshift: 59
band: 59
bnot: 59
bor: 59
btest: 59
bxor: 59
extract: 59
lrotate: 59
lshift: 59
replace: 59
rrotate: 59
rshift: 59
bit32: 59
create: 59
resume: 59
running: 59
status: 59
wrap: 59
yield: 59
coroutine: 59
abs: 59
acos: 59
asin: 59
atan: 59
atan2: 59
ceil: 59
cos: 59
cosh: 59
deg: 59
exp: 59
floor: 59
fmod: 59
frexp: 59
huge: 59
ldexp: 59
log: 59
max: 59
min: 59
modf: 59
pi: 59
pow: 59
rad: 59
random: 59
randomseed: 59
sin: 59
sinh: 59
sqrt: 59
tan: 59
tanh: 59
math: 59
byte: 59
char: 59
dump: 59
find: 59
format: 59
gmatch: 59
gsub: 59
len: 59
lower: 59
match: 59
rep: 59
reverse: 59
sub: 59
upper: 59
string: 59
concat: 59
insert: 59
remove: 59
sort: 59
table: 59
server_scripts: 16
server_script: 16
description: 43
client_scripts: 43
client_script: 43
files: 43
file: 43
dependencies: 43
dependency: 43
__metatable: 2
CreateRPCContext: 33
maps: 1
gametypes: 1
getCurrentGameType: 1
getCurrentMap: 1
changeGameType: 1
changeMap: 1
doesMapSupportGameType: 1
name: 7
race: 5
gameTypes: 9
usjbattle: 1
lovelier: 1
play: 2
ignore_lod_modelinfos: 1
world_definition: 1
modelinfo_deadlock_hack: 1
bounds_arent_cdimage: 1
entity_sanity: 1
static_bound_sanity: 1
odd_wait_deadlock: 1
msgType: 1
hostname: 1
maxplayers: 1
In the meantime, 2 different script environments (LuaGlobal) got removed and created, and the increased rules seem to be specifically for members created in a LuaChunk that I keep between LuaGlobal environments to speed loading up (and yes, I really do have a lot of script environments!).

A short internet search on the DLR RuleCache<T> leaking memory shows that IronPython had a similar issue back in 2009, where precompiling scripts and then reusing them in a different global environment would cause rule leakage... however they resolved it, but I can't find the specific patch that did resolve the issue.

I hope this helps figure out the actual issue, at least. :)
Coordinator
Aug 31, 2014 at 10:51 AM
Edited Aug 31, 2014 at 12:14 PM
I execute following commands:

if you want to test it your self:
NeoCmd
> type = type
>

> type = type
>

> type = type
>

>:cache
==================================================================
SetMember Binders
==================================================================
type: 2

>:env

> type = type
>

> type = type
>

> type = type
>

>:cache
==================================================================
SetMember Binders
==================================================================
type: 4
You see, if you create a new LuaGlobal/LuaTable NeoLua creates a new rule set (OnNewIndex, SetValue) ,that gets added. The get not collected :(. But the good news. The number will not grow over 128 rules.

Why (I am happy, that I founded): I embed in the SetValue-code the reference to the internal values store, and so for every LuaTable + every member a new rule set will be created. The problem is, that the rule holds references to the value store and the CallSite.

Are the growing binders are all values of a LuaTable?
Can you let it grow to the maximum and check out how many memory it consumes?
> for i = 1,1000,1 do
> local t = {}
> t.type = type;
> end;
>

> :cache
==================================================================
SetMember Binders
==================================================================
type: 128
It is may be possible to change the code, and not embed the values, but it might reduce speed.

Edit: It is hard to change the code.
Aug 31, 2014 at 3:05 PM
Hi,

Yes, it indeed maxes out at 128, and the memory usage with my current test case seems within reasonable limits - I was afraid this would end up running out of memory fairly soon.

I still seem to have another issue with memory, however forcing a GC of all generations using 'Force GC' in VS causes the memory usage to remain relatively stable (i.e. outside of the limits of 'needing to worry in a non-stresstest scenario').

Thanks for the assistance.
Coordinator
Aug 31, 2014 at 4:27 PM
Hi,

but when it reaches the 128 cache items, it gets relatively slow. Some time I have to rewrite LuaTable, to get a better performance. The current implementation gets to slow and is to memory intensive.

At the time I have no idea how I will design the class.

Thank you for your valueable hints.

FVI: I am currently merged the new debug pattern ;)
Coordinator
Aug 31, 2014 at 5:23 PM
Edited Sep 4, 2014 at 3:28 PM
Do you plan to do the same tests on version 0.9?

Because, LuaChunk has no Dispose anymore. And I hope all objects are collected. In my tests it worked, but you have a more complex scenario.

Edit: How performs the version 0.9.1?
Sep 9, 2014 at 11:24 PM
Ah, I just got a chance to update to 0.9.2, and it's amazing! Script compilation time is reduced heavily, and best of all, no unneeded rules are kept upon recompiling the script!

Now, I wonder how well it will work when downstream users of my service update to my new service with NLua replaced with NeoLua; until now I've been doing the transition in a branch and nobody outside of me tested it, but especially with the improved debug output I'd almost see it make sense for general use. :)
Coordinator
Sep 10, 2014 at 11:19 AM
Thank you,

this makes my day. I noticed the same, in some cases in my application, I am more then 100 times faster than NLua. Only, the fill process of array is a little slower, the rest is at least as fast as NLua.

Currently, I integrated the lua test suite in my project and found a lot small bugs/incompatilities. When this is done I will remove the beta, and report, what is different and what not.

After this I will start to build the traceline debugger, as an addon.