New Feature: Render targets and rendering to bitmap

Coordinator
Nov 18, 2009 at 1:27 AM
Edited Nov 18, 2009 at 1:34 AM

Up until now, the only rendering available in SilverSprite was Canvas rendering. For each Draw call, a corresponding element would be put on the canvas. While this works pretty well for fairly small numbers of sprites, it falls apart when you're dealing with hundreds or more, especially if they're tinted and all moving around, and partially transparent, and you're creating a lot of them at once. One place where you might run into all 4 of these characteristics is in a particle engine.

To address these issues, along with adding the ability to do additive blending, you can now choose to render to a bitmap instead. In this case all of the rendering and blending is done by the SilverSprite code to a WriteableBitmap. If you create a RenderTarget2D, and choose any surface format except a special SilverSprite surface format of Canvas, you'll be rendering to bitmap for the render target. You can also make the main game canvas use bitmap rendering by specifying a preferred back buffer format in the game constructor:

_graphics = new GraphicsDeviceManager(this);
_graphics.PreferredBackBufferFormat = SurfaceFormat.Rgba32;

Be very careful using bitmap rendering because it can be CPU intensive, but if you really need it you don't have any other real choice. Also rotation doesn't currently work in bitmap rendering and there may be some other issues currently. Non-bitmap text may have issues as well. 

One other use of bitmap rendering can be to get nice sharp edges when scaling. This is especially useful for 8 bit retro grames. If you're doing this, try to scale once using a render target, then use the default canvas rendering for your main game surface and render your scaled render target bitmap to there.

These changes are currently only in the source tree, once I have a good feeling about the stability of the current code I'll cut an Alpha release.

I'll add a sample of bitmap rendering and render targets to the solution soon.

 

Nov 18, 2009 at 3:40 PM
Edited Nov 18, 2009 at 3:41 PM

Hi Bill - thanks for all your efforts, I'm really looking forward to trying out the rendertargets.

In the meantime I'm getting an error with the event wc_OpenReadCompleted (Discovery.cs), as e.Result throws a "The URI prefix is not recognized" exception:

{System.Reflection.TargetInvocationException: An exception occurred during the operation, making the result invalid.  Check InnerException for exception details. ---> System.Net.WebException: An exception occurred during a WebClient request. ---> System.NotSupportedException: The URI prefix is not recognized.
   at System.Net.WebRequest.Create(Uri requestUri)
   at System.Net.WebClient.GetWebRequest(Uri address)
   at System.Net.WebClient.OpenReadAsync(Uri address, Object userToken)
   --- End of inner exception stack trace ---
   --- End of inner exception stack trace ---
   at System.ComponentModel.AsyncCompletedEventArgs.RaiseExceptionIfNecessary()
   at System.Net.OpenReadCompletedEventArgs.get_Result()
   at SilverArcade.SilverSprite.Manifest.Discovery.wc_OpenReadCompleted(Object sender, OpenReadCompletedEventArgs e)
   at System.Net.WebClient.OnOpenReadCompleted(OpenReadCompletedEventArgs e)
   at System.Net.WebClient.OpenReadOperationCompleted(Object arg)}

 

{System.Reflection.TargetInvocationException: An exception occurred during the operation, making the result invalid.  Check InnerException for exception details. ---> System.Net.WebException: An exception occurred during a WebClient request. ---> System.NotSupportedException: The URI prefix is not recognized.

   at System.Net.WebRequest.Create(Uri requestUri)

   at System.Net.WebClient.GetWebRequest(Uri address)

   at System.Net.WebClient.OpenReadAsync(Uri address, Object userToken)

   --- End of inner exception stack trace ---

   --- End of inner exception stack trace ---

   at System.ComponentModel.AsyncCompletedEventArgs.RaiseExceptionIfNecessary()

   at System.Net.OpenReadCompletedEventArgs.get_Result()

   at SilverArcade.SilverSprite.Manifest.Discovery.wc_OpenReadCompleted(Object sender, OpenReadCompletedEventArgs e)

   at System.Net.WebClient.OnOpenReadCompleted(OpenReadCompletedEventArgs e)

   at System.Net.WebClient.OpenReadOperationCompleted(Object arg)}

 

 

I notice that the local string xapFile in GetContentInternal() is being set to an absolute path rather than a relative path, however changing to UriKind.Absolute doesn't seem to make a difference.  Any ideas?

Coordinator
Nov 19, 2009 at 12:45 AM

Ok sorry about that, I commented it out for now. It's not being used yet anyway. Please try again.

Thanks,
Bill

Nov 19, 2009 at 9:24 AM

Thanks for the fast response, it's up and running now.  First impressions are positive - the new rendering system is a big improvement as my engine feels snappier and more responsive.  Bitmap fonts no longer 'bleed' onto the screen as before which I'm very happy about.  Great stuff.

I've not yet got the rendertargets quite working.  My code uses the following to draw to the backbuffer:

 

ScreenManager.GraphicsDevice.SetRenderTarget(0, null);

ScreenManager.GraphicsDevice.SetRenderTarget(0, null);

 

According to MSDNGraphicsDevice.SetRenderTarget should use the backbuffer when the rendertarget passed in is null.  Silversprite doesn't seem to be catering for this.  Is there anything I've overlooked?

Nov 19, 2009 at 2:34 PM

Oops.  Please disregard the last post - silversprite does actually handle the null rendertarget correctly, I'd just cocked up my initialisation.

At the moment the memory usage is shooting up dramatically, up to 2GB within a few minutes.  Just trying to figure out what I'm doing wrong.

Nov 19, 2009 at 3:49 PM

Right, ok.  The GetTexture() method (SilverlightRenderBase.cs) seems to be the culprit.  As a quick test I tried using a single, persistent WriteableBitmap so it wouldn't get instantiated during each call, and the memory issues abated (though not entirely).

The only use I have for RenderTargets is to ensure the game screen is scaled up.  You mention that it's possible to do this to achieve proper, retro-esque (i.e. non-blurry) scaling:

  • When I draw the scaled RenderTarget to a normal backbuffer it is drawn blurry, but performance is fine.
  • When I draw the scaled RenderTarget to a bitmap-rendered backbuffer it is drawn correctly with sharp edges, but performance is hit quite badly (also some weird things happen to DrawString locations).

My question:  Is there any way to draw the scaled RenderTarget properly without using a bitmap-rendered backbuffer?

Nov 20, 2009 at 2:34 PM

Here's a playable example of the Canvas vs Bitmap issues here:  http://www.clarvalon.com/SilverLightTestAwakener4/testpage.html

It uses Canvas rendering by default, but you can press 'B' for bitmap and 'C' for canvas to change the renderer mid-game.

(Ignore the error messages - it's an existing game that I'm in the process of converting over to my engine, so it's largely unfinished).

Coordinator
Nov 20, 2009 at 3:39 PM

Thanks for the feedback, I'll be working on some of these issues this weekend and will post back here with my findings.

Bill

Coordinator
Nov 21, 2009 at 3:17 PM

I just checked in a fix for GetTexture which should help memory usage and cpu significantly, please try it out. I have some other ideas I'm going to try out soon as well.

Nov 21, 2009 at 4:51 PM

The fix works fine, and is a little more elegant than what I'd hacked in  :)

Sorry to pester you with all these posts, but I'm curious as to what Silverlight 4 will mean for the SilverSprite project.  Is it likely the JIT compilation will noticeably improve performance?  And will a trusted, OOB app potentially allow us to use the 360 controller?

Nov 21, 2009 at 5:43 PM
Edited Nov 21, 2009 at 5:44 PM

Clarvalon,

Would you mind if I asked you a few questions about your game?


I was curious, it looks like you are using images for your text. Is that the case? And if so why are you not using SpriteFonts?

Secondly, I have been having a hard time getting different objects to draw in the correct order. Is there a trick you have learned to layer everything properly? Or are you drawing everything in a single DrawableGameComponent with a single SpriteBatch?

Appreciate any information you can give.

Thanks

Nov 21, 2009 at 9:04 PM

Hi Omatase,

The game isn't mine, I'm just porting it to my engine (XAGE) which uses SpriteFonts only.  Not sure what you mean by images.

As for draw order, I don't use the built-in spritebatch layerdepth.  All objects in a room are drawn with a single spritebatch - I iterate through each and draw them in order depending on their Y (or Z) position.  So on the most basic level, the objects at the bottom of the screen are drawn after the objects at the top.  This method seems to be the most reliable across all platforms.

Hope this helps. 

Nov 21, 2009 at 10:12 PM

Bill - Small fix for Drawing Bitmap fonts using Bitmap Renderer (not checked in as I don't have an svn client on this machine).  WriteableBitmapRenderer.cs method DrawString() - currently not using origin and not applying scale.  Changes highlighted in red (makes it consistent with existing Canvas SpriteFont rendering):

---

public override void DrawString(SpriteFont spriteFont, string text, Vector2 position, Color color, float rotation, Vector2 origin, Vector2 scale, SpriteEffects effects, float layerDepth)

{

origin = origin * scale;

if (color.A == 0 && blendMode == SpriteBlendMode.AlphaBlend) return;

if (spriteFont is BitmapSpriteFont)

{

BitmapSpriteFont font = spriteFont as BitmapSpriteFont;

Vector2 pos = Vector2.Zero;

foreach (char c in text)

{

if (c == '\n')

{

pos.Y += font.LineSpacing;

pos.X = 0;

continue;

}

if (font.characterData.ContainsKey(c) == false) continue;

GlyphData g = font.characterData[c];

Rectangle rect = new Rectangle(g.Glyph.X, g.Glyph.Y, g.Glyph.Width, g.Glyph.Height);

Blit(font.SourceData, position - origin + pos + new Vector2(g.Cropping.X * scale.X, g.Cropping.Y * scale.Y), ref rect, color, scale, effects);

pos.X += (g.Kerning.Y + g.Kerning.Z + font.Spacing) * scale.X;

             }

}

else

{

_text.Text = text;

_text.Foreground = new SolidColorBrush(color.ToSilverlightColor());

_textTranslate.X = position.X;

_textTranslate.Y = position.Y;

_bmp.Render(_text, _textTranslate);

}

}

WriteableBitmapRenderer.cs method DrawString() - not using origin and not applying scale.  Changes highlighted in red:
public override void DrawString(SpriteFont spriteFont, string text, Vector2 position, Color color, float rotation, Vector2 origin, Vector2 scale, SpriteEffects effects, float layerDepth)
{
            origin = origin * scale;
            if (color.A == 0 && blendMode == SpriteBlendMode.AlphaBlend) return;
if (spriteFont is BitmapSpriteFont)
{
BitmapSpriteFont font = spriteFont as BitmapSpriteFont;
Vector2 pos = Vector2.Zero;
foreach (char c in text)
{
if (c == '\n')
{
pos.Y += font.LineSpacing;
pos.X = 0;
continue;
}
if (font.characterData.ContainsKey(c) == false) continue;
GlyphData g = font.characterData[c];
Rectangle rect = new Rectangle(g.Glyph.X, g.Glyph.Y, g.Glyph.Width, g.Glyph.Height);
Blit(font.SourceData, position - origin + pos + new Vector2(g.Cropping.X * scale.X, g.Cropping.Y * scale.Y), ref rect, color, scale, effects);
                    pos.X += (g.Kerning.Y + g.Kerning.Z + font.Spacing) * scale.X;
             }
}
else
{
_text.Text = text;
_text.Foreground = new SolidColorBrush(color.ToSilverlightColor());
_textTranslate.X = position.X;
_textTranslate.Y = position.Y;
_bmp.Render(_text, _textTranslate);
}
}

 

Nov 22, 2009 at 3:47 AM

Thanks for the info, is this draw method something you just figured out or is there some place I can go to find this kind of information? I have been having a bear of a time getting everything drawn in the order I want.

Nov 22, 2009 at 9:14 AM

Omatase:  Just something I figured out - even though my room objects have update() and draw() methods, they don't inherit from DrawableGameComponent.  Instead the screen's (based on the xna screen management example) Draw() method loops through all room objects:

 

// Draw all Characters and Objects in room depending on calculated Draw Order (based on Z positions)
            List<zp> drawOrder = GetDrawOrder(ALL_IN_ROOM);
            bool drawItem;
            foreach (zp item in drawOrder)
            {
                // TEST
                if (item.num == 35)
                    System.Console.Write("");
                drawItem = true;
                if (ALL_IN_ROOM[item.num].currentFrameString == null)
                    drawItem = false;
                //if (/*(ALL_IN_ROOM[item.num].currentFrameString != "none") &&*/ 
                if (ALL_IN_ROOM[item.num].isVisible == false)
                    drawItem = false;
                if (ALL_IN_ROOM[item.num].BodyAnimNum < 0)
                    drawItem = false;
                if (ALL_IN_ROOM[item.num].isCharacter == false && (string.IsNullOrEmpty(ALL_IN_ROOM[item.num].currentFrameString) || ALL_IN_ROOM[item.num].currentFrameString == "none"))
                    drawItem = false;
                if (drawItem)
                    ALL_IN_ROOM[item.num].Draw(spriteBatch, ScreenManager.GraphicsDevice, false, false);
            }

    // Draw all Characters and Objects in room depending on calculated Draw Order

            List<zp> drawOrder = GetDrawOrder(ALL_IN_ROOM);

            foreach (zp item in drawOrder)

            {

                    ALL_IN_ROOM[item.num].Draw(spriteBatch, ScreenManager.GraphicsDevice, false, false);

            }

... where ALL_IN_ROOM is a list of all the objects in the room, and the GetDrawOrder() method organises them into an ascending list of the order in which we want to draw them (drawOrder).  Probably not the most efficient method but it works for my purposes in both native PC and Silverlight builds.

 

Coordinator
Nov 22, 2009 at 1:48 PM
clarvalon wrote:

Bill - Small fix for Drawing Bitmap fonts using Bitmap Renderer (not checked in as I don't have an svn client on this machine).  WriteableBitmapRenderer.cs method DrawString() - currently not using origin and not applying scale.  Changes highlighted in red (makes it consistent with existing Canvas SpriteFont rendering):

_bmp.Render(_text, _textTranslate);
}
}

 Great thanks this is checked in now.

Coordinator
Nov 22, 2009 at 1:54 PM
clarvalon wrote:

Right, ok.  The GetTexture() method (SilverlightRenderBase.cs) seems to be the culprit.  As a quick test I tried using a single, persistent WriteableBitmap so it wouldn't get instantiated during each call, and the memory issues abated (though not entirely).

The only use I have for RenderTargets is to ensure the game screen is scaled up.  You mention that it's possible to do this to achieve proper, retro-esque (i.e. non-blurry) scaling:

  • When I draw the scaled RenderTarget to a normal backbuffer it is drawn blurry, but performance is fine.
  • When I draw the scaled RenderTarget to a bitmap-rendered backbuffer it is drawn correctly with sharp edges, but performance is hit quite badly (also some weird things happen to DrawString locations).

My question:  Is there any way to draw the scaled RenderTarget properly without using a bitmap-rendered backbuffer?

No, there really isn't any other option I know of to scale the image without anti-aliasing. I was trying to do some with pixel shaders but couldn't quite get it right. This may still be possible. It looks like maybe pixel shaders in WPF and Silverlight don't honor the settings if I set them to use point filtering. I'll need to do a bit more research on this, pixel shaders aren't something I know very well.

One thing I'm curious on if you can try it is that you can use a Canvas surface format for your render target, and then your un-scaled rendering will use the old rendering functionality, and as long as your game's backbuffer is using the new bitmap rendering it should scale up properly. I'm just wondering if this performs better/worse/the same for you as using a bitmap render target.

Nov 22, 2009 at 3:12 PM

I hadn't thought of that.  I've just tried it and, whilst I really ought to put some sort of metric in place to measure this properly, visually it seems less choppy than using a purely bitmap based rendering (whilst still being slower than canvas based), though a few visual glitches remained.  I think going forward I'll build in some visual quality options into the Silverlight Build, i.e.:

  • Low Quality (Canvas backbuffer and canvas rendertarget) - Fast but anti-aliased, visual glitches.
  • Medium Quality (Bitmap backbuffer and canvas rendertarget) - Slower but only a few visual glitches.
  • High Quality (Bitmap backbuffer and bitmap rendertarget) - Slowest but no visual glitches, on par with native PC and Xbox.

Thanks for all your help.

Coordinator
Nov 22, 2009 at 4:27 PM

Have you tried setting the MaxFrameRate in the <object> tag in the HTML? You could probably be able to set it to 30 or so for your game without major noticeable change and it will significantly cut down the CPU usage.

Nov 22, 2009 at 6:48 PM

Doesn't seem to help.  The game logic update seem to be linked to the framerate - I tried setting the maxFrameRate to 4 and it all very very slowly, rather than at normal speed but choppy as you'd expect (regardless of renderer).

I'm satisfied with the performance for the time being.  It runs at 100% on a dual-core machine, it would just be nice to be viable on older hardware too. 

Dec 20, 2009 at 9:19 PM

When using bitmap rendering (SurfaceFormat.rgba32) for the rendertarget, the alpha channel doesn't seem to be quite right.

You can see this on the second set of menus (handwritten, grey background) here:  http://www.clarvalon.com/Xage/Games/Awakener/

When the final menu options fade in, you can change the visual quality using L, M and H keys (Low, Medium and High, as per a few posts above).  On High, which uses a bitmap rendertarget and is on by default, you can see a black outline around the menu options that isn't there when using a canvas rendertarget (as per Low and Medium settings).  It appears as if the alpha channel is being ignored for each pixel that is not completely transparent.

This is using the latest built (37371).  Is it possible I've overlooked some setting?

Coordinator
Dec 24, 2009 at 12:31 PM

Can you send me one of the graphics that fades in? That's where I can see the issue best.

Dec 24, 2009 at 2:36 PM

Here's the original texture for the menu objects:  http://www.clarvalon.com/Temp/OriginalTexture.png

Heres a side-by-side comparison to highlight what the alpha problem is with Bitmap RenderTargets:  http://www.clarvalon.com/Temp/RenderTargetAlphaProblem.png

Coordinator
Dec 24, 2009 at 3:58 PM

Are you including the png or an xnb version in your game? Just want to make sure I can replicate the problem properly.

Dec 24, 2009 at 4:21 PM

The XNB version (http://www.clarvalon.com/Temp/Intro_objects.zip).

Coordinator
Dec 24, 2009 at 9:23 PM

Ok thanks, are you including the XNB or the PNG in your Silverlight project?

Dec 25, 2009 at 11:36 AM

I have a separate app that pre-prepares all the textures as XNB and then zips them up.  These zip files are then included in the Silverlight project.

Coordinator
Dec 26, 2009 at 7:20 PM

Can you try it again, I found out my alpha blending was completely wrong. It should also perform a bit better now.

Dec 27, 2009 at 10:27 AM

The latest build not only fixes the alpha problems but the performance is noticeably better.  When not debugging, the bitmap-based rendering is now almost the same speed as Canvas on my atom netbook, which is excellent.  I wasn't expecting a fix so soon, given the time of year - really excellent stuff, thanks Bill.

Coordinator
Dec 27, 2009 at 2:21 PM

Fantastic, I'm working on something else that will improve the render to bitmap performance even more. Hopefully it will be ready sometime in the next couple of weeks, I'll keep you updated.

Coordinator
Dec 29, 2009 at 3:45 AM

I checked in part of what I'm working on, and this part may help you significantly. If nothing changes between frames with the bitmap rendering, nothing is redrawn. I want to take this a step further and only redraw parts of the screen that have changed, but this isn't working quite right yet.

Dec 30, 2009 at 12:04 AM

It does seem faster, though it is a little choppy.  I think this is due to the way the character's walkcycle works, updating the frame at specific intervals, so whenever the next walking frame is drawn it slows down.  This may be mitigated by your proposed improvements (and it all still runs absolutely fine on a dual-core machine).  I've updated with the latest build:  http://www.clarvalon.com/Xage/Games/Awakener/

One of these days I'll get around to plugging in an fps counter so I can begin to backup these observations with actual numbers!

Dec 31, 2009 at 5:27 PM

I've spotted a new problem using the latest build (37890).

XAGE utilises a screen system based on the GameState Management Sample.  The pause menu appears on top of GamePlay screen.

For some reason, the GamePlay screen is no longer visible in the pause menu.  At first I thought this was some bug I'd introduced, however it only ever occurs when bitmap rendering is used.

To recreate:  http://www.clarvalon.com/XAGE/games/Awakener/  - get into the game, and press escape to pause.  Press L, M and H to change visual quality.  For Low (canvas only), the GamePlay screen is still visible when paused, as expected.  For Middle and High (bitmap), the GamePlay screen is no longer visible.  

I've no idea why this is happening.  I'm fairly certain this only started happening since using 37890.

 Brandhall Dental Car
 4 Brandhall Road, 68 8DP
 0121 552 1333
 Karen
 NHS - YES 
 Appointment: Monday 11th 2

 

Coordinator
Jan 17, 2010 at 6:19 PM

Can you try the latest? I think I may have found what was causing this.

Jan 19, 2010 at 12:44 PM

I've tried it with 38755 and the problem remains (plus the "Click to Activate" start screen is no longer visible, itself a member of the MenuScreen class).

Coordinator
Jan 19, 2010 at 2:05 PM

So just to be clear, is the main issue that you expect a partially transparent overlay for the pause screen? I'm not completely familiar with how the game state manager works but do you have multiple SpriteBatch.Begin and End calls in a single draw loop? Also do you call GraphicsDevice.Clear multiple times in a single draw loop? Just trying to narrow this down a little bit.

Jan 19, 2010 at 2:42 PM

Yes.  The correct behaviour can be seen on the above link when using canvas only rendering (L).

All screens use the same ScreenManager, which itself is a DrawableGameComponent.  This ScreenManager has a single spriteBatch.  

Each screen has their own Draw() method that clears the ScreenManager.GraphicsDevice and begins and ends ScreenManager.SpriteBatch.