Previously, we were scaling down the source image to fit into
the smaller sourceRect, instead of cropping at the original scale.
This broke the background textures in Fancy Pants World 4 Part 2,
as the scaled-down output image resulted in a smaller rectangle
being returned from 'getColorBoundsRect'
We now crop the image by properly constructing the UV-coordinate
transformation matrix. We were also using the wrong value for the
'destPoint' y coordinate, which I fixed.
This slightly changes the image output of two tests - the new images
now more closely match the Flash output.
* Pixels with 0 alpha are not affected by color transforms.
* Color channels should be clamped to the 0-255 range.
* A color transform with only an alpha multiplier of >1 has no
effect.
When using the bitmap.wgsl shader for normal rendering, we need
to saturate immediately after applying the color transformation
to reproduce Flash Player's behavior. This makes the (possibly
transformed) alpha value get multiplied by a in-range color,
instead of a potentially out-of-range color.
However, Stage3D just applies a no-op color transformation,
and should only saturate at the very end
(not after the intermediate division by the original alpha value).
To support both of these requirements, I've added in a new
`early_saturate` ifdef that controls when we apply 'saturate'.
We then compile the shader twice (once with early_saturate=true
and once with early_saturate=false), and use the two versions
in the right pipelines.
We could use a simpler shader for Stage3D - however, it can't just
be a plain copy, as we need to apply the viewport transformation.
For now, I'm re-using the shader code to keep things simple. If
this becomes a performance issue in stage3d, we could revisit this.
This was a leftover from before we started usiung vec4 everywhere
for compatibility with AGAL. There are a few specific opcodes that
don't need extension, but it doesn't depend on the destination
register.
In the process, I fixed a bug where we were clearing the depth
and stencil buffers with the incorrect value.
This makes Fancy Pants World 4 Part 1 playable to completion
(though there are still some rendering issues that need
to be fixed).
We don't need to perform a sync when getting the width/height,
getting or setting the 'disposed' status, or uploading to
a Context3D texture.
The Context3D change (using `copy_texture_to_texture` instead
of relying on the CPU pixels) has the added advantage of avoiding
a validation error when our source image row length isn't aligned
to `COPY_BYTES_PER_ROW_ALIGNMENT`
This dramatically speeds up the Fancy Pants World 4 loading time
(on a branch with my XML prs merged). Without this change, my
machine spends around 10 seconds on a blank white screen after
clicking 'Play'. With this change, the time spent on that screen
is reduced to around 1-2 seconds.
None of these formats can currently be implemented
correctly with wgpu, so we just use Rgba8Unorm instead.
The handling of opaque compressed textures is a little
sketchy - it should work for 'normal' SWFs that upload
an opaque BitmapData, but we might need to manually
adjust the alpha values if
* Bump bitflags to 2.0.0
* Sprinkle Clone, Copy, Eq, PartialEq, and Debug derives where needed
* Call `bits` on bitflags, as it is now a method
* Switch from `from_bits_truncate` to `from_bits_retain` on bitflags where needed
* Bump h263-rs for the bitflags 2.0.0 dependency
As part of porting to bitflags 2.0.0, see:
https://kodraus.github.io/rust/2022/10/07/bitflags2.html#upgrading-to-2x
We were ignoreing 'data32PerVertex'.
To make the code clearer, I've renamed the variable to
'data32_per_vertex', and made it a 'u8' (as it has a maximum of 64)
Webgl doesn't support BGRA textures, so this lets us use
Stage3D textures on the web backend. As a bonus, this speeds up
uploading an BitmapData to a Context3dTextureFormat.BGRA texture,
since we no longer need to change the format before copying.
This makes Solarmax2 playable on the web backend.
This is a very large diff, but most of it comes from test files and
output.
This PR ads partial support for the following Stage3D shader features:
* Normal (square), rectangle, and cube textures
* Varying and temporary registers
* Lots of opcodes
The combination of these allows us to get a raytracing program
fully working in Ruffle. I've included it as image test.
Currently, this test is very slow (about 90 seconds on my machine),
as the code I'm using (https://github.com/saharan/OGSL) includes
its own shader language and compiler. THe raytracing demo
first compiles its own shader language to AGAL, and then starts
rendering the scene.
Limitations:
* Many opcodes are still unimplemented
* Most non-default texture options (e.g. mipmaps) are not implemented
When rendering to an offscreen texture for `Bitmapdata.draw`,
we first render to a temporary frame buffer, and then copy the contents
of the frame buffer back to the target texture. However, this results
in blend modes being incorrectly applied - for example, rendering with
BlendMode.SUBTRACT will subtract against the framebuffer (which starts
with each pixel as 0x00000000), instead of the previous BitmapData
contents.
To fix this, we now use our texture target as the frame buffer
when performing `render_offscreen`. This ensure that we blend
over existing pixels (taking into account the `blendMode` provided
in the `BitmapData.draw` call).
When multisampling is enabled, we use a copy pipeline to copy
the existing contents of our texture to a fresh multisampled frame
buffer (the non-multisampled texture target becomes our resolve buffer).
* Take two: Delay reading image back from render backend using `SyncHandle`
This allows us to avoid blocking immediately after a `BitmapData.draw` call.
Instead, we only attempt to use the `SyncHandle` when performing an operation
that requires the CPU-side pixels (e.g. BitmapData.getPixel or BitmapData.setPixel).
In the best case, the SWF will never explicitly access the pixels of
the target BitmapData, removing the need to ever copy back the render backend
image to our BitmapData. If the SWF doesn't require access to the pixels immediately,
we can delay copying the pixels until they're actually needed, hopefully allowing
the render backend to finish processing the BitmapData.draw operation in
the backenground before we need the result.
Now that the CPU and GPU pixels can be intentionally out of sync with
each other, we need to ensure that we don't accidentally expose 'stale'
CPU-side pixels to ActionScript (which needs to remain unaware of
our internal laziness). We now use a wrapper type `BitmapDataWrapper`
to enforce that the `SyncHandle` is consumed before accessing the
underlying `BitmapData.
* core: Skip GPU->CPU sync for source and target BitmapData during draw
* Introduce DirtyState enum
Previously we were removing the first occurrence of the invalid
0xFFD9FFD8 byte sequence in JPEG data, but this would break the
JPEG if it happened to contain this byte sequence elsewhere (for
example, EXIF data). Instead, properly parse the JPEG markers
searching for the invalid marker sequence.
Fixes#8775.
This makes `Bitmap` delegate to `BitmapData` for
all of the bitmap-related information (handle, width, and height).
As a result, we now unconditionally store a `BitmapData` in `Bitmap`.
As a result, swapping the underling `BitmapData` instance will
automatically change the properties (and rendered image) of a `Bitmap`.
This required some refactoring in the render backends in order to
get access to a `BitmapHandle` through `BitmapData`.
There were two issues:
1. We were accidentally calling `as_any` on `handle,` rather than
`handle.0`
2. Calling `as_any` can invoke the wrong implementation, depending on
what traits are in scope. We want the method implemented on the
underlying type (`RegistryData`) to be used, but if `Downcast` is
explicitly imported, then we appear to invoke it on the trait object
`dyn BitmapHandleImpl` itself (using the fact that trait objects
themselves implement `Any`). We now explicitly call the generated
method on the trait object, which avoids this issue.
`BitmapHandle` now holds `Arc<dyn BitmapHandleImpl>`.
This allows us to move all of the per-bitmap backend data into
`BitmapHandle`, instead of holding an id to a backend-specific
hashmap.
This fixes the memory leak issue with bitmaps. Once the AVM side of a
bitmap (`Bitmap`/`BitmapData`) gets garbage-collected, the
`BitmapHandle` will get dropped, freeing all of the GPU resources
assoicated with the bitmap.
This PR implements core 'stage3D' APIs. We are now able
to render at least two demos from the Context3D docs - a simple
triangle render, and a rotating cube.
Implemented in this PR:
* Stage3D access and Context3D creation
* IndexBuffer3D and VertexBuffer3D creation, uploading, and usage
* Program3D uploading and usage (via `naga-agal`)
* Context3D: configureBackBuffer, clear, drawTriangles, and present
Not yet implemented:
* Any 'dispose()' methods
* Depth and stencil buffers
* Context3D texture apis
* Scissor rectangle
General implementation strategy:
A new `Object` variant is added for each of the Stage3D objects
(VertexBuffer3D, Program3D, etc). This stores a handle to the
parent `Context3D`, and (depending on the object) a handle
to the underlying native resource, via `Rc<dyn
SomeRenderBackendTrait>`).
Calling methods on Context3D does not usually result in an immediate
call to a `wgpu` method. Instead, we queue up commands in our
`Context3D` instance, and execute them all on a call to `present`.
This avoids some nasty wgpu lifetime issues, and is very similar
to the approah we use for normal rendering.
The actual rendering happens on a `Texture`, with dimensions
determined by `createBackBuffer`. During 'Stage' rendering,
we render all of these Stage3D textures *behind* the normal
stage (but in front of the overall stage background color).
We only called `get_bitmap_pixels` when creating a `BitmapData`
for an SWF-provided `Bitmap`. We now store the initial pixels
in `Character::Bitmap`, and use them to initialize a `BitmapData`
when needed.
This lets us simplify the wgpu backend, which no longer needs
to store a `Bitmap` object. In addition to saving space for
`BitmapData` objects that lack an SWF `Bitmap`, this will make
it easier to move data from `bitmap_registry` into `BitmapHandle`
itself.
Currently, we rely on ShapeTessellator being able to get a BitmapHandle
without a RenderBackend. With the upcoming BitmapData refactor,
we will always need a RenderBackend to get a BitmapHandle, which creates
borrow-checker issues in ShapeTessellator (which is stored in a
RenderBackend).
To solve this, we split BitmapSource.bitmap into two methods -
BitmapSource.bitmap and BitmapSource.bitmap_handle. ShapeTessellator
continues to use BitmapSource.bitmap, and uses the u16 bitmap id
instead of a BitmapHandle. The BitmapSource.bitmap_handle method
is used inside each render backend to convert the id to a BitmapHandle,
avoiding borrow-checker issues.
This is the first part of the Stage3D implementation, and can
be reviewed independently.
Stage3D shaders use the Adobe Graphics Assembly Language (AGAL),
which is a binary shader format. It supports vertex attributes,
varying registers, program constants (uniforms), and texture sampling.
This PR only implements a few parts of AGAL:
* The 'mov' and 'm44' opcodes
* Vertex attributes, varying registers, program constants, and 'output'
registers (position or color, depending on shader type)
This is sufficient to get a non-trivial Stage3D program
running (the rotating cube demo from the Adobe docs).
The output of `naga-agal` is a `naga::Module`. This can be passed
directly to wgpu, or compiled into a shader language using
a Naga backend (glsl, wgsl, SPIR-V, etc). The test suite
output WGSL files, and uses the 'insta' crate to compare against
saved files on disk.
Currently, the only real way to write AGAL bytecode is using
the Adobe-provided 'AGALMiniAssembler' flash class.
This class assembles the textual reprentation of AGAL into
the binary format.
To make writing tests easier, I've added a 'agal_compiler' test, which
can easily be modified to add more Agal textual assembly.
This PR fixes a numbe of interconnected bugs:
* We weren't consistently uploading a dirty BitmapData to the render
backend before drawing to/from it.
* BitmapData.draw should *not* add a fill color - it should draw over
the current contents of the BitmapData
* After drawing to a non-transparent BitmapData, we need to manually
set the opacity back to 255 for each pixel (the drawing process
takes transparency into account, but the opacity information is
thrown away at the end).
Change `Bitmap::new()` to accept a `ruffle_render::bitmap::Bitmap`
directly, instead of `width`, `height` and `bitmap_handle`. As a
consequence, all `RenderBackend::register_bitmap_*` methods are no
longer necessary - we can use `ruffle_redner::utils::*` to obtain
a `ruffle_render::bitmap::Bitmap` right before calling `Bitmap::new()`.
And make it generic, as a first step towards making it a general-purpose
data structure for the whole codebase. Some potential replacements are:
* `BoundingBox` in `render/src/bounding_box.rs`.
* `BoxBounds` in `core/src/html/dimensions.rs`.
* Parameters to a bunch of `BitmapData` methods in
`core/src/bitmap/bitmap_data.rs`.
Main changes:
* Merge `ColorTransformParams` into `ColorTransformObject`, as it's only relevant for AVM1.
* Make `BitmapData::color_transform` work with a generic `ColorTransform`, which uses fixed-point
arithmetic.
Note that Ruffle still calculates color transforms slightly different from Flash. This is probably
caused by inaccuracy of the current `ColorTransformObject` to `ColorTransform` conversion and/or the
`ColorTransform` application logic itself. Since this requires further research, it'll be fixed in a
future PR.
As usual, also bump its helper crates (`js-sys`, `web-sys` and
`wasm-bindgen-futures`) to the latest versions.
Due to https://github.com/rustwasm/wasm-bindgen/pull/3031, use the
`serde-wasm-bindgen` crate as a replacement to the deprecated
`JsValue::from_serde` function.
In several cases, the current code seems preferable to the
code required by `clippy::bool_to_int_with_if`. Let's suppress
this for now to get the build passing, and decide later if this
is something that we want to enable.
When rendering offscreen, we want the resulting image to use
premultiplied alpha, since the image will be stored in a texture.
However, when capturing an image in the exporter or test framework,
we want to use straight alpha, so that the resulting image can
be saved as a PNG.
Previously, we incorrectly used straight alpha everywhere, resulting
in incorrect output when using BitmapData.draw with transparency.
We now check if a BitmapData has been disposed by checking
for a zero width or height (which cannot happen otherwise).
As a result, we no longer need the 'disposed' field on the AVM1
BitmapData object.
* avm2: Implement `BitmapData.draw` for `wgpu` backend
This method requires us to have the ability to render directly to a
texture. Fortunately, the `wgpu` backend already supports this in
the form of `TextureTarget`. However, the rendering code required
some refactoring in order to avoid creating duplicate `wgpu` resources.
The current implementation blocks on copying the pixels back
from the GPU to the CPU, so that we can immediately set them in
the Ruffle `BitmapData`. This is likely very inefficient, but will
work for a first implementation.
In the future, we could explore allowing the CPU image data and GPU
texture to be out of sync, and only synchronized when explicitly
necessary (e.g. on `getPixel` or `setPixel` calls).
* Rename `with_offscreen_backend` to `render_offscreen` and use Bitmap
* Don't panic when backend doesn't implement `render_offscreen`
Fixes some issues with our winding # calculation which would cause
incorrect results for hitTest.
* The convention for handling an intersection at endpoints was
not the same between lines and bezier curves.
* The bezier curve winding # function was not properly handling
some cases where the curve was strictly y-monotonic.
* Simplify the code a bit so that ray-curve intersections are
returned in a consistent order based on upward/downward crossing.
It was only used to make structs `#[derive(gc_arena::Collect)]`, and
generally it doesn't make much sense that `render` needs to be GC-aware.
So instead annotate `render` fields in `core` with `#[collect(require_static)]`.
Reverts #7267
The image tests for the upcoming 'DisplayObject.stageRect' support
differ between Linux and Windows, so we need this support again.
To avoid the Linux filename churn that we previously encountered,
we now only include the platform and graphics backend in the filename
(e.g. `expected-linux-Vulkan`). This may result in some unexpected
'mismatched image' test failures if GHA updates to a version of Lavapipe
that changes rendering output, but this should be relatively easy to
notice.
Previously, the viewport height and width were stored in
both `Stage` and the `RenderBackend`. Any changes to the viewport
dimensions (e.g. due to window resizing) needed to be updated in both
places to keep our handling of the viewport consistent.
This PR adds a new `ViewportDimensions` type, which holds the
width, height, and scale factor. It is stored inside the
`RenderBackend` impl, and is retrieved using the newly added
method `RenderBackend.get_viewport_dimensions`. After a `Player`
has been constructed, any code that needes access to the viewport
dimensions will ultimate go through this method.
Unfortunately, `Stage` needs to use the viewport dimensions
in `build_matrices`. Therefore, any code modifying the viewport
dimensions should go through `player.set_viewport_dimensions`,
which ensures that the stage matrices are rebuilt after the render
backend is updated.
The captured WGPU texture uses premultiplied alpha.
This image gets saved as a PNG, so it should use straight alpha.
Note that all of our current image tests have 'alpha = 1.0' for all
of the pixels, so this currently has no effect.
Each render backend keeps track of a stack of BlenModes,
which are pushed and popped by 'core' as we render objects
in the displaay tree. For now, I've just implemented BlendMode.ADD,
which maps directly onto blend mode supported by each backend.
All other blend modes (besides 'NORMAL') will produce a warning
when we try to render using them. This may produce a very large amount
of log output, but it's simpler than emitting each warning only once,
and will help to point developers in the right direction when they
get otherwise inexplicable rendering issues (due to a blend mode
not being implemented).
The wgpu implementation is by far the most complicated, as we need
to construct a `RenderPipeline` for each possible
`(BlendMode, MaskState)`. I haven't been able to find any documentation
about the maximum supported number of (simultaneous) WebGPU render
pipelines - if this becomes an issue, we may need to register them
on-demand when a particular blend mode is requested.
This basically reverts #5737 and #6458 for the WebGL backend, which
regressed a bug where setting the style `display: none;` to a Ruffle
player logged many WebGL warnings to the console. This happened
because `renderbuffer_width` and `renderbuffer_height` were set to zero,
leading to problems when trying to pass them to WebGL APIs.
Avoid such situation by ensuring that `renderbuffer_width` and
`renderbuffer_height` are at least `1`, exactly as done before.
Also add a comment that explains why `.clamp()` isn't used.
Fixes#1264.
Though https://github.com/rust-lang/rust-clippy/pull/8355 has been
merged, it seems to still report false-positives on nightly channel.
For now just fix the instances reported by stable clippy, and keep
`needless_borrow` allowed.
Aaron Hill