A mesmerizing burst of circles (Silverlight 5 & XNA)

Let’s give some love to Silverlight and in particular Silverlight 5’s 3DApi, which happens to be XNA.

I want to draw something cool on this XNA DrawingSurface BUT then also replicate it in DirectX. I’ve honestly never done a DirectX/C++ project so this will be very risky and a completely new experience for me….

This post will concentrate on the Silverlight 5 part, and the next post will be the DirectX part.

The IDEA

I want to experiment mixing UIElement’s that sit in the visual tree and sprites that sit in the XNA “DrawingSurface” . Below is a very simplified pic of how the UI will be layered…

• XAML UIElements sitting in the top layer (Retained Mode UIElements)

   

  1. As the user moves the mouse around the screen I will draw a nice ‘UserControl’ that gives us the x,y position of the mouse in a visual way.
  2. This UserControl will also have a “Dot” to represent the exact mouse location

• “DrawingSurface” Sprites  (Immediate Mode Sprites)

   

  1. A bunch of purple sprites will follow the mouse around the screen
  2. When the user clicks someone on the screen a bunch of green sprites will burst at that location
  3. The circles fade over time
  4. The circles move with random velocities, in random directions from the mouse-point

The Top Layer

What we want is a special shape that follows the cursor around as you move the mouse. And this shape needs to hold the x,y position of the mouse. Also this is all rendered in the XAML’s visual tree.

Blend ships with some awesome shapes, one of those is exactly what we need

I cannot begin to tell you how useful the shapes are in Blend, and I really hope that a lot of these blend libraries are available in WinRT-XAML apps in beta.

Silverlight and Blend are mature, WinRT isn’t and that is my main concern with WinRT. I really hope we don’t need to wait 3 more years for WinRT to reach the maturity of Silverlight 5 and Blend 4

Basically I am going to use this “Rectangular Callout”, add it to the visual tree then move it around as the cursor moves. I’m also going to add a small dot to signify the center of the mouse-point.

To animate these 2 controls (Ellipse, Callout) I am going to use a storyboard, which has some great easing rules built into it which will allow me to give it a nice eased motion.

The code that actually wires up the MouseMove and triggers the storyboard is self-contained in a class I called “CalloutCursor.cs” . I won’t go into the details of how that works, just know that it executes the storyboard, plugging in values where the mouse currently is. It also has the calculations to make sure the pointy bit of the callout is in the correct orientation.

So this is what the TOP layer looks like when all the above pieces are running … Pretty simple, and that’s the beauty of Silverlight & Blend, it’s so easy to do stuff like this (once you know your way around that is) ..

 

 

The Bottom Layer

This is where it get’s interesting … We are going to use the XNA bits in Silverlight 5 to draw a burst of circles where the current mouse position is.

First step is to place the XNA surface in the XAML where we will render our sprites.

Notice that we wired up the “Draw” event of the DrawingSurface. This triggers 60 times a second and it’s during these events that we draw the sprites on the surface.

For those that haven’t written much XNA code before there is this concept of a “scene”, we wrap the XNA animation inside of this scene. I have created a scene class that wraps all the logic of rendering the sprites when the mouse moves..

 

The actual SceneBurst class is here : Scene.Burst.cs

In the scene there is this concept of a sprite, and in my case I created a sprite called “BurstingCircles”, and as the name suggests it wraps up all the logic to draw the sprites on the surface.

I create 2 instances of these “BurstingCircles” sprites, one follows the mouse around and the other appears where the mouse is clicked

And these sprites are actually drawn in the “DRAW” method..

 

Now the actual “BurstingCircles” sprites code is here if your interested :

Sprite.BurstingCircles.cs

The BurstingCircles sprite is made up of 100 circles that randomly spray in 360 degree directions from a central point. These 100 circles appear one after the other, not all in one go.

Each circle is made up of 2 triangle primitives which is made up of 6 vertexes (each triangle is made up of 3 vertex).

Notice that I use a VertexPositionTexture to represent each vertex, the operative word here is “texture”. We are going to use a texture to draw on each triangle.

The texture that we are going to use is dynamically created. This is pretty cool … Silverlights WriteableBitmap lets us take a snap of a visual element and create a bitmap out of it, it is this bitmap that we are going to use as the texture

 

The texture I dynamically create is based on an ellipse and i create it with a dynamically defined colour. You can do some damn interesting things when we mix the power of Silverlight’’s mature framework and the XNA immediate mode graphics pipeline.

The draw routine is pretty straight forward, generate the 100 circles over time and re-calculate the velocity information during each cycle. This is pretty standard when dealing with pipeline graphics rendering … It’s really low level, to the metal … love it

I should briefly mention that we use simple (not complex) pixel & vertex shader’s. There’s nothing special about them, their purpose is to get pixels onto the surface and draw non-complex vertexes of the triangles.. No morphing of complex pixel effects for us yet !!!

 

So this is what the bottom layer looks like, it’s pretty simple once you get the hang of how the XNA pipeline works and how to draw in this pipeline..

 

The Install Experience

The 3DAPI (XNA) bits require GPU and Elevated Trust. So getting the user to give you these permissions is tricky.

I’ve gone with some simple graphics and I won’t go into how I did this as it’s pretty common knowledge how to wire up these setup/install screens, if you really want to see code just download the sample

This is what the user sees to ask for their permission to run the GPU demo and to install the app as an “out of browser” app. I chose to make this OOB because it’s easier to give this elevated trust than keep it in browser and getting the user to go into the dialogs and give manual permission to the Graphics card.

These are the Silverlight standard dialogs explaining that we want “elevated trust” for our app …

This is what the APP looks like in browser once it is installed ..

This is what the APP looks like when it is running Out Of Browser, having been manually installed by the user.

 

 

The Result

Here is what it looks like when everything is pulled together

 

Or try out the demo, you need Silverlight 5 to run it

 

 

The Silverlight 5 XNA Api’s are damn awesome, the CPU hit on this is less than 5% and memory footprint is small. My mind is exploding with ideas around 3D animations to compliment Visual Tree UI elements.

I really hope that every developer out there that has invested time to learn Silverlight/WPF and/or XAML, takes the time to learn XNA and/or graphics pipeline immediate mode programming. It seriously opens up your mind in a whole new direction

The next step is to reproduce this in DirectX-WinRT .. off to do just that  … stay tuned!!!

Win8 Concept : App Manager

So this concept design for an “APP MANAGER” poped up on the interwebs and I must say I like it, clean and totally inline with the Win8 metro style of the rest of the OS.

Concept App Manager

reference : Win8China

A shimmering wall of sequin’s to compare WinRT XAML & Silverlight 5

Let’s get a bit creative this week.

I have this vision in my head of a wall of glittering sequin’s, you know what a sequin is right?! like what was big in the ‘disco’ era, and still is big today ?!..

For this post I want to start out simple and create a basic shimmering wall of sequins , nothing too complex!

My ulterior motive for doing this is to compare , what I’m guessing will be, GPU/CPU intensive animations between Silverlight 5 & WinRT XAML.

SILVERLIGHT 5 Approach

Let’s first do this in Silverlight …

STEP 1 : create a ‘sequin’ control.

Starting with Blend I visually drag a couple of ellipses and a rectangle for the background. And to add interaction there’s a “Storyboard”

There are basically 3 UI elements in the visual tree for an individual sequin. The storyboard gives the sequin it’s glitter effect.

NOTE : It’s important to point out that this process was a 100% visual task. With a “designer” hat on I would tweak gradients, pixel position vectors, add new visual elements as I saw fit. If I were in a different mood I may have made 6 UIElements, or maybe 12 , it all depends on my mood and how detailed I wanted to go. I’ll come back to this point later !!! ….

Step 2 : Create a wall of sequin’s

Taking the sequin control from above I layout hundreds of these elements to form a wall. To this was simple, I created new control “scr_Board” and used the “WrapPanel” layout control to wrap hundreds of these ‘sequin’ controls. I did the creation of the sequin’s in code and it’s not clean code, this is a demo after all  

I did say that the code was not very pretty right .. Yes that is me calling into the sequin control from the board control setting & starting the storyboard animations for the sequin.. … I don’t recommended you do this in your projects, i’ve done it to make a point that I will come back to later in this post!!

So I’m creating 2584 sequin instances and loading those in the WrapPanel then starting the storyboard animation to initiate the glitter like effect for each sequin.

Also in the code you see me setting storyboard parameters with random values, that gives the random glitter effect..

Below is a quick video clip of the Silverlight 5 demo running.

 

Step 3 : Performance Tune the xaml

The hardest part of building solutions is tuning it for performance. And for something that is visually rich and uses a lot of animations you NEED to do A LOT of tuning.

There are 2514 sequin’s controls that are all animating individually. And this is a very simple demo. This is how this non-optimized solution performs. That’s around 74% CPU , 115MB Memory Consumed

Now let’s start playing with CacheMode, dictating to the GPU what surfaces to combine etc. So setting it at the root of the sequin visual tree, this is the performance improvement we get… That’s around 51% CPU , 156MB Memory Consumed

Because the visual tree for a sequin is very simple, 4 elements in all, I’m going to go ahead and set CacheMode on each and show you the results..

I normally wouldn’t’ go through every permutation of visual element and set the CacheMode on them…

The process I take is go through the visual tree and work out what makes sense to combine as a surface on the GPU and set the CacheMode on those. It may require me to tweak the visual tree by moving elements around and/or removing non-necessary ones. Also bare in mind if particular visual elements are involved in animations and are often changed then they probably don’t qualify to be cached.

This goes back to what I said earlier around how the UI was created by a designer like person using a visual tool like cider or blend. It’s easy to create very deep visual trees BUT as we can see above if we want a very performant application we need to take care with the design of the visual tree, OR spend time refactoring it before deploy.

So the best we can get is around 48% CPU with 160Mb MEM footprint. I won’t argue with you that 48% is terrible …. There’s more we can do to bring that down especially with how we animate and call storyboards, or our actual design of the storyboards.. The ultimate solution would be to go with “immediate mode” rendering using the XNA API … I will definitely do this in a future POST!!!

 

WinRT XAML Approach

Let’s now take the Silverlight app and port it to WinRT XAML …

STEP 1 : create a ‘sequin’ control.

Taking the SL5 seqin control to WinRT I discover that there was no “RadialGradientBrush”.

I tried creating my own “RadialGradientBrush”, extending the ‘GradientBrush’, BUT I kept getting XAMLTypeInfo COM exception errors when I tried referencing my custom class in xaml …

I believe the issue is related to this reported bug by Andy Beaulieu – “XamlTypeInfo.g.cs Generation Error”

In the end I had to settle for a “LinearGradientBrush”, ill revisit this when Beta arrives .

Step 2 : create a wall of sequins

There is no WrapPanel in WinRT so I took a Silverlight one and directly ported it over. Worked first go, nice and easy. I found the code on CodeProject .

Another issue I found was that ‘x:name=[name]’ elements in a control were no longer accessible from a parent control because the logic around ‘Protection levels’ has changed. eg. My sequin controls that had Storyboards and DoubleKeyAnimations defined could no longer be set/called from the parent. This is a great thing in my opinion

 

 

Another issue that arose was around DependencyProperties.Register() and it takes strings rather than types. This has been fixed and hopefully we’ll get these changes in beta very soon! (below is a screen shot of the issue I just mentioned above around DependencyProperty)

So here ‘s a quick video of the SL5 app converted to WinRT XAML with the few changes mentioned above.

What you’ll notice is that there appears to be a long pause on the black screen after the splash screen loads, that’s when the 2584 sequins are being dynamically created, initialized and added to the WrapPanel. I need to investigate this, as I mentioned this was just a simple port over from SL5 to WinRT XAML.

Also what’s not clear from the video is how buttery smooth the xaml metro app runs as you move between other metro apps and desktop. It is infinitely smoother than the Silverlight app running with other desktop apps, I can definitely say that the GPU acceleration of metro style apps is having a positive impact on performance

 

Another annoying issue that popped up is whenever I closed the app and killed it manually, there would always be an orphaned process created. There are so many of these weird issues popping up in this developer preview of Win8, and as a developer you will see a serious amount of these!

 

Another issue that popped up is that sometimes the app process would stay in the “Background Processes” section of the Task Manager and sometimes it would correctly appear in the “Applications” section.

 

Step 3 : Performance tuning

WinRT XAML also has CacheMode, you can set UIElement’s to be “HardwareBitmapCached” or “SoftwareBitmapCached”

Without any tweaking the CPU usage is at around 45% and the Memory around 130MB.

With tweaking I can get the CPU down to 30-35% and Memory to around 105MB

I know that that’s still a lot of % CPU , and I doubt the Windows Store would let any app that behaved as such BUT in my eyes that is brilliant that it can get to 30 in the first place. There is a lot of CPU draining code in there, and as I mentioned previously this type of animation is screaming to be done using a retained mode solution, XNA/DirectX …. Hopefully we’ll get that in Beta

What is a worry is that when the metro app is pushed to the background, it’s still running & consuming resources excessively. It appears that I the APP maker would need to detect this move to the background and carry out my own code to stop my app . It makes sense that I would need to do this HOWEVER a very small part of me would have wished that maybe MS could have added performance tuning smarts around storyboards and stopped running ones when a metro app goes into the background .  That’s why I chose to put all animations in storyboards because technically they lend themselves to being better tooled, tweaked, analyzed & orchestrated etc. by the CLR and or the OS . I hopped that the platform could do smart things to them at the appropriate time, BUT I thought wrong! (picture below is the metro glittering sequins demo app running in background for over 15 minutes)

Another very interesting issue that came up with WinRT compared to Silverlight is that animations seemed to run so much faster in WinRT. I had to slow things down just to get comparable glittering

 

Conclusion:

You can definitely achieve great results with Silverlight 5 if you take care with your visual tree and know how to optimize and use the provided tooling to debug/trace  performance issues.

BUT WinRT definitely is showing signs of being an amazing GPU Accelerated framework that let’s you achieve levels of performance greater than anything Silverlight can offer today. And that’s not to say that Silverlight is bad, it’s all about using the right technology for the job.

If I were to build a client solution  I would do this using an immediate mode approach, XNA or DirectX. Hopefully I’ll do just that in a future post

 

 

Let’s end with the glittering sequins again .. this is it again in WinRT ( I should point out that the UX definitely suffers because it’s using a LinearGradientBrush instead of a Radial one, yes those things make a huge difference . And it’s so much smoother when running as the video I took dropped frames hence the glittering is very jittery )

 

From now on all my future post’s will have an element of creativity in them, I have to get back to my roots

 

[UPDATED – 22nd January 2012]

As some of you have rightly pointed out, the code changed in the Win8 WinRT XAML version for the sequin, i.e. using a “LinearGradientBrush” instead of a “RadialGradientBrush”, will have an impact on performance in the Silverlight version. This is definitely true and i have run a simple test to see what difference this makes …. The improvement was around 2-4% reduction in CPU BUT the MEMORY pretty much stayed the same.

When I removed a whole “Ellipse” from the visual tree this reduced the MEMORY footprint by around 30-40MB .

Where Windows 8 WinRT-XAML definitely out performs sis when you are moving between apps and windows, it’s seriously buttery smooth.

All eyes are now on Windows 8 Beta

Looking at Silverlight adoption statistics (for fun)

These stats are purely for fun, digest these figures with a huge serving of salt! . 

All these stats come from RIASTAT’s and I’ve pulled them from other people’s posts. There are other statistic collection sites that I also like to use, like StatOwl.

The stats collected are from ALL INTERNET CONNECTED DEVICES, even if Silverlight doesn’t work on that device e.g. Android, iOS etc. It basically sums up ALL devices that connect to the internet and detects if Silverlight is installed. It just so happens that the majority of internet connected devices are Windows OS’s hence the large Silverlight/Flash adoption.

As mobile devices increase I would expect the Silverlight and Flash stats to diminish, thou we aren’t seeing that yet. Which could mean that even thou mobile devices are increasing in adoption, so are PC’s and MAC’s.

 

NOV 2010

reference: Post : Kelly White

Silverlight 2+3+4 sits around 54%

MAY 2011

reference: Post : Code Project

Silverlight 2+3+4 sits around 72%

JAN 2012

reference: RIASTATS January 2012

Silverlight 2+3+4 sits around 76%

Now if you look at just Windows 7 machines you see a bigger penetration of Silverlight.

Silverlight 2+3+4 sits around 86%

A huge percentage of Windows 7 devices have Silverlight, that’s is brilliant!!

And what’s even more exciting to see, the latest version of Silverlight 4 (85.4%) has a greater adoption than the latest version of Flash 11 (72.1%). Bare in mind Silverlight 5 only came out <1 month ago.

And with major movie studios demanding Silverlight to stream their content (eg. Netflix/LoveFilm in the UK require Silverlight) I do believe that Silverlight adoption can only go up (at least on the Windows & MacOS ecosystems that is)

Anyway as mentioned above these stats are just for fun, it shows that Silverlight adoption is still trending up  (NOT DOWN) and even if MS aren’t spending billions for Olympics streaming rights it’s still holding it’s own

Go Silverlight !!!!  

Understanding the Gyrometer data from WinRT

I spent a little time this week trying to make sense of the data coming back from the Gyrometer in the WinRT api’s from my Samsung slate. This is what I discovered…

But before I show you my findings I wanted to say that it has been a complete pleasure using the WinRT sensor api’s. I’ve used the Wp7 ones and can say first hand that WinRT’s are much cleaner and you can achieve much more with fewer lines of code.

In my opinion the true power of WinRT is that you will be able to easily orchestrate streams of heterogeneous sensor data coming from multiple sensors delivering phenomenally richer UX’s . For me this is what I am most excited about

Sample code :

Metro style app designs that are starting to excite me!

I’m really starting to like what I’m seeing in metro designs that are leaking. They’re starting to really show me what level of re-imagining Microsoft and partners/developers are making for their apps. More importantly it’s showing me the level of risk people are taking with Metro.

It’s very risky designing your apps for a metro/muti-sensor first world. Exciting times us developers/designers live in.

And I honestly am starting to see possibilities of metro style apps in the business/enterprise world. As long as these apps compliment desktop apps or can exist side-by-side in the desktop that is. I really hope MS shows us the “desktop” story!

Most of these pictures aren’t of a sufficient quality to be able to zoom into, and are generally sourced from 2 sites

1. ithome.com

2. win8china.com

 

 

I ❤ C# 2

This post follows on from my previous post “I ❤ C#” which discusses interesting recent MS jobs that mention a lot of Managed stack work at very low system levels.

It was pointed out to me that  the “Technical Strategy Incubation Team” has been in existence , that we know of, as early as April 2011 – post reference. Possibly even earlier …

What we know of that incubation team is that it consists of some Midori folks (Mary Jo Foley – Microsoft’s Midori: Who’s on the all-star roster? )

What we can’t directly link but definitely sounds related is that this team probably has A LOT to do with Redhawk including the bit’s we’ve found in Windows 8 (SLR100.dll / WCL*.dll) .. – (Mary Jo Foley – Microsoft codename ‘Redhawk’ lives… in Windows 8 )

What’s interesting with these latest job posts is the scope of the work the incubation team is now doing. Seems like the groundwork and foundation is now in place to build amazing managed features. Things like “structured storage” or “cloud solutions” are being mentioned as areas being built on top of this new radically re thought OS stack.

What’s also interesting is the explicit mention of radical rethinking of UI/UX frameworks and the associated graphics stack/pipeline by this team. Seems like a lot has happened under the covers that hopefully will be in the Windows 8 beta dropping in 2012 Q1 (JAN-MAR).

What’s also interesting is I have been closely following Windows 8 Server as well as System Center 2012 . I did a series of tweets that covered an MS Distinguished Engineer & Lead Architect “Jeffrey Snover ” discussing Windows Server 8 – Storage Solution (I recommend you watch the video)

Apparently the storage services are .NET services running in Windows Server 8, they have a wealth of cmdlets for automating them via WMIv2/PowerShell). I wonder if this at all has anything to do with the “Incubation Team” and their work with cloud storage, as well as the new strength of C#. – job reference structured storage subsystem

This Incubation Team has a lot of juicy managed work, I’ll keep my eyes out for news/jobs relating to them for a long time