This also results in a far reduced role for `ReturnValue`, since I also took the liberty of removing most of it's use. Furthermore, I also made it apply equally to native and AVM2 code, which ensures all native implementations of methods don't double-borrow.
In AVM1, `ReturnValue` was actually removed entirely, because it's not needed. I attempted to do the same, but the fact that we're currently embedding `ScriptObjectData` in native objects means that we need it for virtual properties. Otherwise, virtual property implementations will see locked objects, which is bad.
This commit breaks the build: we still need to tell `Avm2` how to turn ABC traits into our own internal `Trait<'gc>`, `Class<'gc>`, and `Method<'gc>` types. We also need something to track which traits have already been instantiated, because `callstatic` would otherwise reinstantiate the trait in a different scope. (In fact, I think it *does* do exactly that right now...)
The intention is to completely replace all usage of `Avm2XYZEntry` with `Class`, `Trait`, and `Method`. This will allow runtime-provided global class traits to coexist with those provided by user code.
Inspired by Dinnerbone's PR doing the exact same thing to AVM1.
On AVM2 we have a bit of a subtle issue: the base implementation of `set_property_local` and `init_property_local` *must* return `ReturnValue`s to avoid double-borrows. Each implementation of `TObject` must resolve them before returning.
The reason for this is that, in AVM2, `toString` and `valueOf` are not defined on the classes or prototypes of `Function` or `Class`. Instead, they use the `Object.prototype` versions of those functions. Ergo, string and primitive coercion are inherent object methods (the ones that get `[[DoubleSquareBrackets]]` in the ECMA standards). In Ruffle, our equivalent to `[[DoubleSquareBrackets]]` methods are methods on the `TObject` trait, so we're adding them there.
This mechanism will make implementing boxed value types (ala AVM1's `BoxedObject`) easier, too.
We also add some reasonable defaults for `ScriptObject` and `FunctionObject` which will appear on objects, functions, and classes.
Private names now return `false`, and we run any names through trait lookups. This also means any namespace resolution can fail now, in case we need to throw a `VerifyError`.
I have... significant reservations with the way object enumeration happens in AVM2. For comparison, AVM1 enumeration works like this: You enumerate the entire object at once, producing a list of property names, which are then pushed onto the stack after a sentinel value. This is a properly abstract way to handle property enumeration.
In AVM2, they completely replaced this with index-based enumeration. What this means is that you hand the object an index and it gives you back a name or value. There's also an instruction that will give you the next index in the object.
The only advantage I can think of is that it results in less stack manipulation if you want to bail out of iteration early. You just jump out of your loop and kill the registers you don't care about. The disadvantage is that it locks the object representation down pretty hard. They also screwed up the definition of `hasnext`, and thus the VM is stuck enumerating properties from 1. This is because `hasnext` and `hasnext2` increment the index value before checking the object. Code generated by Animate 2020 (which I suspect to be the final version of that software that generates AVM2 code) initializes the index at hero, and then does `hasnext2`, hence we have to start from one.
I actually cheated a little and added a separate `Vec` for storing enumerant names. I strongly suspect that Adobe's implementation has objects be inherently slot-oriented, and named properties are just hashmap lookups to slots. This would allow enumerating the slots to get names out of the object.
Previously, we were treating ES4 classes like syntactic sugar over a prototype chain (like ES6 classes); e.g. each declared trait was set in the given prototype and then property look-ups happened as normal.
This already caused problems with virtual properties, which could be partially-defined in subclasses and required careful checks to make sure we stopped checking the prototype chain on the *correct* half of the property.
However, this is a hint of a larger problem, which is that ES4 classes don't actually define anything on the prototype chain. Instead, the instance itself constructs class properties and methods on itself. This allows things like methods automatically binding `this`, which isn't included in this commit but will be implemented really soon.
The prototype chain still exists even on pure ES4 classes, due to the need for backwards compatibility with ES3 code. Object, for example, still defines it's methods as prototype methods and thus there needs to be a prototype chain to reach them. I actually could have gotten away with using the prototype chain if AS3 *hadn't* retained this "legacy" detail of ES3 allowing this class/prototype distinction to leak out into upcoming tests.
We still actually use the prototype chain for one other thing: trait resolution. When we look for a trait to install onto an object, we pull traits from the prototype chain using a special set of `TObject` methods. This happens in opposite order from normal prototype lookups so that subclassing and verification can proceed correctly.
`super` somehow became even harder to implement: we now actually construct the parent class so we can get traits from it, which is going to complicate method binding as mentioned above.
The previous system for handling setters would execute the setter and then return a value to indicate whether or not the caller needed to resolve a stack frame. However, no caller of `Property.set` actually did this. Ergo, errors in setters and getters would not resolve up the stack at the correct time.
This problem also exists in AVM1 but is far less noticable as AVM1 only has two very uncommon runtime errors and very few movies use `throw`.
Normally, `set_property` only affects the object it was called on, which makes sense: otherwise, we couldn't override values that originate from a class prototype without accidentally monkey-patching the prototype. However, virtual setters only exist in prototypes and need to be accessible from child objects.
The solution to this is to have a specific method to check if a virtual setter exists. Virtual setters are then resolved through the prototype chain. If no virtual setter exists, then the reciever object is handed the value.
Note that we always use the `reciever` object rather than `self` so that `setsuper` can work correctly. In `setsuper`, we resolve the base class, and then set properties on it with the actual object in question as it's reciever. If a virtual setter is called, it will get the actual object it should be manipulating; and otherwise, prototypes will not be modified or consulted.
This required the reintroduction of dedicated reciever parameters to `Object.get_property_local` and `Object.set_property`, which I had removed from the AVM1 code I copied it from. It turns out being able to change the reciever was actually necessary in order to make super set/get work.
Also, implement a method table that method traits can optionally add themselves to.
Also also, add the ability to invoke a method without a `this` object. This required a non-trivial refactoring of the activation machinery, and changes to the signature of `NativeFunction`, and all native AVM2 functions.
This allows the AVM to declare classes, which necessitated some refactoring to avoid double-borrows or having to do something "magic" that would dodge virtual properties.
I'm writing all this code assuming that classes and traits are syntactic sugar around ES3-style prototype chains on function objects. Hence, `FunctionObject` is still our workhorse object type for implementing typing.
David Wendt