This is inspired by Dinnerbone's similar PR on the AVM1 side, where the Action half of that VM's `Executable` was reduced from 128 bytes to 16 by shoving it in a `Gc`. This won't be as dramatic but should still save some memory.
In fact, it should save a *lot* of memory in bytecode execution, where thanks to the previous commit's rebase, we now need to clone the current method once *for each instruction executed*. That is terrible, but should stop now.
This also results in a far reduced role for `ReturnValue`, since I also took the liberty of removing most of it's use. Furthermore, I also made it apply equally to native and AVM2 code, which ensures all native implementations of methods don't double-borrow.
In AVM1, `ReturnValue` was actually removed entirely, because it's not needed. I attempted to do the same, but the fact that we're currently embedding `ScriptObjectData` in native objects means that we need it for virtual properties. Otherwise, virtual property implementations will see locked objects, which is bad.
Previously, we were treating ES4 classes like syntactic sugar over a prototype chain (like ES6 classes); e.g. each declared trait was set in the given prototype and then property look-ups happened as normal.
This already caused problems with virtual properties, which could be partially-defined in subclasses and required careful checks to make sure we stopped checking the prototype chain on the *correct* half of the property.
However, this is a hint of a larger problem, which is that ES4 classes don't actually define anything on the prototype chain. Instead, the instance itself constructs class properties and methods on itself. This allows things like methods automatically binding `this`, which isn't included in this commit but will be implemented really soon.
The prototype chain still exists even on pure ES4 classes, due to the need for backwards compatibility with ES3 code. Object, for example, still defines it's methods as prototype methods and thus there needs to be a prototype chain to reach them. I actually could have gotten away with using the prototype chain if AS3 *hadn't* retained this "legacy" detail of ES3 allowing this class/prototype distinction to leak out into upcoming tests.
We still actually use the prototype chain for one other thing: trait resolution. When we look for a trait to install onto an object, we pull traits from the prototype chain using a special set of `TObject` methods. This happens in opposite order from normal prototype lookups so that subclassing and verification can proceed correctly.
`super` somehow became even harder to implement: we now actually construct the parent class so we can get traits from it, which is going to complicate method binding as mentioned above.
The previous system for handling setters would execute the setter and then return a value to indicate whether or not the caller needed to resolve a stack frame. However, no caller of `Property.set` actually did this. Ergo, errors in setters and getters would not resolve up the stack at the correct time.
This problem also exists in AVM1 but is far less noticable as AVM1 only has two very uncommon runtime errors and very few movies use `throw`.
The previous system primarily relied on `Executable` to automatically start and continue a super chain. This works, but only for class hierarchies without *override gaps* - methods that override another method not defined by the direct superclass of the method. In that case, the override method would be called twice as the `base_class` was moved up one prototype at a time, which is wrong.
The new system relies on the call site to accurately report the prototype from which the current method was retrieved from. Super calls then start the resolution process *from the superclass of this prototype*, to ensure that the already-called method is skipped.
It should be noted that the proper `base_class` for things like `callmethod`, `callstatic`, `call`, `get`/`set` methods, and other call opcodes that don't use property look-up are best-effort guesses that may need to be amended later with better tests.
To facilitate `base_proto` resolution, a new `Object` method has been added. It's similar to `get_property`, but instead returns the closest prototype that can resolve the given `QName`, rather than the actual property's `ReturnValue`. Call operations use this to resolve the `base_proto`, and then resolve the method being called in `base_proto`. The existing `exec_super` method was removed and a `base_proto` method added to `exec` and `call`.
Also, implement a method table that method traits can optionally add themselves to.
Also also, add the ability to invoke a method without a `this` object. This required a non-trivial refactoring of the activation machinery, and changes to the signature of `NativeFunction`, and all native AVM2 functions.
David Wendt