Implement "GUI Layer" based Scaling Options to better scale text and other objects

[Silver-Streak]

Hello All,

This is a moderately large request, so I’ve put most of the historical info at the bottom. At a high level, I’d like to request the following:

Proposal

• A new layer type, named “GUI Layer” be added to the IDE. Users should have a button to add a GUI layer just like they do to add a Lighting layer.
• A new Game Resolution Resize mode named “GUI Resize with Asset Zoom” or something similar to project properties.
• If this mode is selected:
  • Another set of resolution fields should be displayed underneath for “Game asset resolution width” and “Game asset resolution height”
  • Another field should be displayed underneath those named “Scale type” with “Scale based off Width” or “Scale based off Height”
• A backend calculation occurs to calculate the Asset Zoom and the GUI Scale/Zoom to be used in later calculations.
  • Asset zoom calculates based off the Game’s Current Resolution divided by the defined Game Asset Resolution Width/Height.
    • e.g. Game project is set to 1280x720p resolution. The game is running and hasn’t resized. The game asset resolution is defined as 320x180. The user defined to scale off height. 720/180 = 4x Asset zoom. If the player resized the game to 1920x1080, asset zoom updates to (1080/180) 6x zoom.
  • GUI Scale or zoom calculates based off the the Game’s current resolution compared to the Game’s project defined resolution.
    • e.g. Game project is set to 1280x720p. The player has resized the game to 1920x1080. The GUI Zoom is calculated as (1080/720) 1.5x
• All NON-GUI Layers are automatically zoomed by the Asset Zoom level. This makes them zoom in as if they were made at the game project resolution. I have completed testing and this has no quality or performance loss compared to using scaling methods.
• All objects on GUI Layers have their positions, sizes/scales, font sizes, etc, automatically multiplied by the GUI Scale/Zoom. These sizes are recalculated upon resizing the game/changing the resolution.
  • e.g. The game project is set to 1280x720. The player’s HP/Max HP text is set to be at 100 X 150 Y. The font size is 60 px. The player has resized the game to 1080p. The game window is actually increased to 1080p instead of scaled. The HP/Max HP text is now positioned at 150 X 225 Y (original * 1.5), the font size is now 90 px. To the player, the UI is actually positioned exactly as it was before proportionally, the game is just bigger.
• All events that occur to GUI Layer objects take this GUI Zoom/Scale into account.
  • e.g. The user making the game always only has to enter 100 X 150 Y. The backend should see these values as 100 * GUI Scale Factor X, 150 * GUI Scale Factor Y.
• Expressions should be added to obtain the project resolution, the asset zoom factor, and the GUI Scale/Zoom factor
• Pixel Rounding/Position Rounding should be added as a project setting.
  • Regardless of this setting, Text on the GUI Layer should always have it’s font size rounded.
• An option to disable/hide the toolbar in preview should be added. Due to how Pixi/GDevelop forces the resolution to include the toolbar, it gives an inaccurate representation of games in fullscreen.

Benefits

• This fixes all font scaling issues with any text on a GUI layer, since they’re not being scaled, just resized and repositioned.
• This makes it so game assets can be at a MUCH smaller resolution than the native resolution, allowing for much smaller projects and in many cases better memory and CPU utilization.
• Implementing this eliminates a lot of very difficult math for newer users in order to replicate this functionality, and makes it seamless in the background.
• This should be renderer agnostic if GDevelop ever switches to/adds other renderers.

Drawback

• We’ll need clear documentation both in the engine and in the wiki that explains asset resolution vs game resolution (I’m happy to build this out if this is implemented)
• This requires addition to both the IDE and backend code. I know it is possible since I’m doing it in the event system, but I don’t know the complexity.

Example Project
GDevelop GUI Layer Example by Silver-Streak (GUI Layer Proposal Project.zip)
Here’s the example project folder, I’ve added a ton of comments on the general events (yellow comments), and more details on the proposal (red/pink comments).

Example Compiled Game
GDevelop GUI Layer Example by Silver-Streak (GUI Layer Example Build.zip)
Default Platformer Controls (Directional Pad)
Press P to resize to 1600x900 window.
Press O to resize to 1280x720 window.
Press M to fullscreen the game.
Press N to restore to window.

Background
I’ve been doing as much research as I can and wracking my brain to try and figure out the difference between how GDevelop scales assets and how other cross-platform/HTML5 engines do.

For other PixiJS engines (Construct and Phaser), it seems like they’ve heavily branched from Pixi to establish their own methods. I don’t think changing renderers is a reasonable request or a relatively easy task, so I kept digging.

I started looking into how Defold does it, and how Game Maker Studio’s system handles it. What I’ve discovered is that both of them implement “GUI Layers” and “Non-GUI Layers”. GUI Layers are rendered at native resolution, while Non-GUI Layers can be any resolution and are scaled/zoomed in to match the native resolution.

With that knowledge in mind, I’ve been plugging away at the event system trying to figure out if, and how, this could be done in GDevelop. I’ve now confirmed that it can be done, so I’d like to request this be implemented as an actual engine feature. I feel that the math involved is pretty complex, and would be difficult for new users to replicate on their own. However, if this is implemented, users can have perfect pixel fonts, text, and other objects regardless of their game size.

References
Defold GUI Layers: GUI scenes in Defold
Game Maker Studio’s GUI Layers: Gui Layer Secrets

[convictedweirdo]

After seeing the project you made, I agree this would be a great feature to have as trying to make a UI work in GDevelop can be frustrating as you end up having to rely on work-arounds and hacks.

[Eliath_x]

I just checked your UI Layer example. After checking the three, how do you see the situation? I’ve read the drawbacks (and benefits) and which seems most suitable is the Pinning one, it doesn’t need as much work on maths as the Text Layer example and keeps things into a single layer.
Hopefully this is being worked in the future, specially for pixel art games that cannot use Text objects due to the scaling issue it has at the moment.

[4ian]

Thanks for this investigation I’ve not understood everything so I’ll have to re-read everything again.
Your example shows though that it’s possible to get crisper text than the default rendering.
Note though that when resizing the game to a sufficiently large size, we can see that the rendering is not pixel perfect:

Which seems to indicate that this maybe more a workaround/optimization/improvement than a definitive solution (for example compared to a Bitmap Text using a Bitmap font generated from a custom texture?).

(quick note too: Construct and Phaser are both not based on Pixi.js. Phaser used to do, but is not since years now. Construct never was if I’m not mistaken).

Ideally I would want to ensure that we have a definite pixel perfect solution before implementing anything complex that could risk create more confusion than what we already have. For example, if the text rendering is found to be never pixel perfect in Pixi.js, then all hopes to get a pixel perfect font are lost - there is no way to make this perfect… Unless you use a Bitmap Text with a custom font created from a texture. (something to implement in the Bitmap Text).
A workaround can be to create a font with a very large font size and then making it smaller than what it should be - so that it is “crispier” (i.e: “almost” “pixel perfect”) but that’s an illusion that will fail if you make the window sufficiently large.

I think there was this discussion on a Pixi.js issue: can the text rendering be pixel perfect for fonts made in a pixel-art style? If not, we should probably invest our time in Bitmap Text.

[Silver-Streak]

Hey @4ian

Can you let me know what resolution that was at?

That seems like for some reason it scaled, not resized (the Font size should update in the text, but it seems to still be at 64).

I was able to use my solution to get up to 4k without issue, so I’m confused at what combination of items led to your screenshot.

Edit: Here’s a snapshot of it at 4k. Still seems pixel perfect to me?

That said, the GUI Layer method seems pretty common on numerous different engines.

Double edit: Oh, some discrepancy may be because I am not rounding any of the values in my example. I’ll upload an updated one that should fix that.

[Silver-Streak]

@4ian

Here’s an updated project with rounding enabled. Visual appearance seems to be the same at 1080p and 720p, which is great since they’re not integer resolutions of one another. Also thanks for calling out Construct, it looks like the info I found was someone released a PixiJS addon for construct rendering with C2.

Project: https://easyupload.io/yw5me0
Example Windows Build: https://easyupload.io/mw3oxg

Please note: when I’m saying Pixel Perfect, I mean that the text is rendered the same at any resolution, not the complete elimination of sub-pixel rendering. That is entirely dependent on the font and apparently how Canvas rendering works on pixel fonts, not the engine, so if the font has a very minor amount of sub-pixel rendering at that size, it always will.

e.g.

In this example, native res is 720p, if I scale it in GIMP to 300% (4k) it has visible subpixel rendering.
If I render at 1080p and scale it in gimp to 200% (4k) it still has subpixel rendering.

Note: This subpixel rendering is true regardless when you scale up the image from it’s native size, but when at their native size (720p/1080p), to the player they’ll appear identical.

The GUI Layer proposal above will help for both pixel fonts and non-pixel fonts, so I still think it’s a viable solution, and seems to match up with how other engines do GUI layers.
(This will also have an added benefit of allowing users to use much smaller assets while still having higher quality GUI even if they’re not worried about text rendering)

Edit: Slight correction, while subpixel rendering is a factor of the fonts themselves, it’s also related to how Canvas and PixiJS render text differently than they do images (as we’ve seen in the numerous examples I’ve done on the github issue). Unless we’re somehow going to full on not use any PixiJS/Canvas text rendering and do something else, this seems like the best way to make all text appear “normal” regardless of resolution, or at least match the quality the text appears at it’s native resolution.

[Silver-Streak]

Just wanting to check in with @4ian again on this one to see if you’ve had a chance to look at the update.

The other thing I’ve tested is we can easily implement the “increase fontsize 4x, scale text object / 4” into this method as well, if needed. (Although we still need scale options for BBText to make it work there).

That isn’t a real replacement for the GUI Layer functionality (smaller asset size vs native resolution GUI), but is another function that could be added (probably another toggle in game properties “use scaling correction for text” or something) for certain fonts that need it.

[4ian]

I took a look at the latest version of your game example just now.
I think I get the basic idea:

• keep the resolution of the game to be resolution of the window,
• zoom in the base layer so that you always have 180 pixels as height,
• and for the UI layer, multiply the position/sizes/font sizes by a factor to keep them visually the same size.

The last point allowing to keep “crisp” fonts because you’re not asking Pixi.js or the browser to scale up the rendered text. I say “crisp” but not pixel perfect because this method will never get pixel perfect results because the text rendering is always made with “subpixels” aliasing it seems.

So while you’ve proved that we can do this entirely with GDevelop itself, the feature request is “let’s make this easier for people by having “GUI Layer”” (correct me if I’m wrong :)).

I’m for now unsure about how to balance “making it easy for the user”, “making it understandable” and “don’t add a new layer (pun intended :)) of complexity in the rendering engine”. I’m a bit afraid that adding a do-it-all “GUI layer” option could be seen as a bit magical (“why is this even necessary, can’t I just put text wherever I want??”) or complex (“wut why is this engine changing the font size on its own?”).

I wonder if we could make this feature by a combination of smaller features, like:

1. An option to have the zoom of a layer adapted to the window size.
2. Improve the anchor behavior to do what you did manually for the object positioning.
3. And something (an option?) to ensure the text font size is changed - or maybe doing the trick of “fontsize 4x, scale text object / 4” would work?

The idea would be for each of these 3 smaller features to be something working on its own.

Let me know what you think, and if you think it would work?

The other thing I’ve tested is we can easily implement the “increase fontsize 4x, scale text object / 4” into this method as well, if needed. (Although we still need scale options for BBText to make it work there).

That could be an option too yeah, as mentioned we could have it as a “Make text rendering in high resolution (useful for pixel fonts)” toggle, so that you would not need dynamic font size changes when the window is resized (up to a point, but well 4x is already pretty big).

[Silver-Streak]

So while you’ve proved that we can do this entirely with GDevelop itself, the feature request is “let’s make this easier for people by having “GUI Layer”” (correct me if I’m wrong :)).

Yes, absolutely. I want to make accomplishing what I did easier for everyone else in the future (and myself as well. :D)

The idea would be for each of these 3 smaller features to be something working on its own.

Let me know what you think, and if you think it would work?

High level: Yes, I think that these could be broke into separate options. Although I think merging them into one “GUI Layer Settings” may be easier for people to understand, splitting it is totally doable with some caveats.

Benefits of splitting it up:

1. Users can pick and choose which functions to use as needed.
2. Could be rolled out in different PRs over time

Risks of Splitting it Up:

1. Some of the items may not add any value to users until all items are implemented
2. More complicated for users to understand how to use together effectively.
3. Splitting them off means we will still likely need an even larger tutorial on how to use them together to build a GUI, and may make it harder for people to understand vs 1 feature set.

All of the above said, so long as the same goals are accomplished, I’m happy to write up tutorials either way once these items are implemented to help out users of the engine.

More detail:

I wonder if we could make this feature by a combination of smaller features, like:

1. An option to have the zoom of a layer adapted to the window size.

This could work, but I think the issue you run into here is that you have 2 different type of zooms needing to happen:

• Asset Zoom up
  • Asset layers need to be scaled to fit the game resolution. Not sure how you could represent this without outright asking the user to define their asset layers “faux-native” resolution.
• GUI “Zoom” (really just GUI asset resize/reposition)
  • Either up/down depending on Game resolution in properties vs current resolution, GUI needs to be kept to the same “visual” size regardless of the game resolution.

The GUI Layer idea was to accommodate both of these (GUI layers are kept at window resolution and objects automatically re-positioned/resized. Asset Layers are scaled up based off defined faux-native resolution), but I think this could work if the ease of use is addressed, although this does mean that players would have to do this in every scene. (Still easier than manually recreating the events, I’d assume)

2. Improve the anchor behavior to do what you did manually for the object positioning.

Definitely, even if it’s just a toggle of “Keep object position proportionally in this position regardless of resolution”, which would basically be the same as what I’m doing in the example (store original x/y, multiply it by the new resolution factor, reposition)

3. And something (an option?) to ensure the text font size is changed - or maybe doing the trick of “fontsize 4x, scale text object / 4” would work?

I think this one is tricky. There are 2 different things at play.

1. You want text to appear proportionally the same size to the rest of the GUI regardless of the game resolution.
   • So if 60px font at 720p, it needs to be 90px (60px*1.5x larger resolution, rounded) font at 1080p so it’s still the same proportional size of the GUI.
2. You want the text to appear as crisp as possible.
   • The GUI layer method in the example does this, but even that can be improved by taking the current font size * 4, then scale the text/bbtext object by 0.25

#2 isn’t absolutely necessary since #1 already gives a really clean appearance, but you COULD accommodate both of them. Have some kind of “Keep text proportionally the same size on all resolutions” toggle AND a separate “High quality text scaling” toggle.

What would happen is if both are checked, it would do the 90px calculation for font size > THEN *4x > THEN scale it back down 0.25. ((Text PX Size * Resolution factor) * 4) > Scale 0.25
The biggest thing is that by doing this you’d probably also need to automatically apply the anchor behavior to insure the x/y position stays the same, I think?

No matter what we’d need a way for people to apply this to BBText (and BitmapText) since BBText does not have any events related to scaling currently, unless there is not an intent to keep parity between native engine capabilities and user event capabilities.

[Silver-Streak]

Hey all.

A few people on the discord had been asking about the examples as the easyupload links expire after thirty days. I’ve gone ahead and updated the original post with the most recent example project and build, linking to itch.io which should keep the links active in perpetuity.

Please note that I do not know if this feature is being adopted or being considered, unless @4ian or others have had more recent conversations on it, but I hope the examples help people evaluate the events and math required to complete this.

[Silver-Streak]

Just an update, it sounds like the newest version of Cocos Creator came out, and added UI Layer functionality OOTB similar to what my proposal is above, so at minimum it sounds like this is something other engines are doing. (calling it “design resolution” vs “screen resolution”)

Here’s a link to their documentation for reference, if @4ian wants to take a look. https://docs.cocos.com/creator/manual/en/ui/multi-resolution.html

[krunkster]

I apologize for bumping an old thread, but when I was searching for an answer to this problem, this is where I ended up.

I was convinced that there was some sort of “aliasing” enabled on the fonts that I couldn’t disable, but based on the mention of “bitmap” fonts in the conversation above and having just read the most recent release notes, I realized that at least the “aliasing” has been solved I guess.

For any other searchers who come later:
In the latest beta 5.0 b110 there is now support for Bitmapped Fonts, and using the BMFont tool you can bake any pixel font at a particular font size and use the bmfont objects instead of text objects and you will end up with pixel-perfect text rendering instead of the anti-aliased version of the TTF.

[Silver-Streak]

Yes, Bitmap font solves the Blurriness that my GUI layer helps resolve. Note that it does not solve the “distortion” you get if you aren’t using integer based resolutions.

(e.g. with bitmap font, going from 720p to 1080p will still make the fonts look “weird” without the GUI layer idea above.

With Bitmap Font, and starting at a smaller res that scales appropriately, like 640x360, will allow you to scale up without distortion into 720p, 1080p, 1440p, etc.)