Dynamic backgrounds

update : Demo code works for Release Preview (RP)

Don’t be afraid to create crazy backgrounds for your managed metro apps. This is a quick post on how to do just that using some of the techniques I’ve been posting about.

For this post ill stay simple BUT the next post I promise to go crazy with the colors, textures, geometries and especially the animations

 

Idea

The idea is simple, you have your DirectX layer underneath your XAML layer and you render dynamic things on the Dx layer. These dynamic things for this demo are simple geometries, colors and radial gradients.

 

Demo screen shots

This is a plain white background with a radial gradient painted ontop of it using D2D (via SharpDx) and the Tiles are rendered on the XAML layer above the D2D layer.

Press on the blue tile

Press on the Red tile

Press on the Green tile

Press the Orange tile

 

Demo video

 

Demo code

UPDATED : Now works for Release Preview

Conclusion

Don’t be afraid to give your managed metro apps some crazy DirectX rendered backgrounds, think outside of the box and experiment. These are after all consumer apps that need to wow the user.

SharpDx gives us some great tools to do crazy things, I hope to show you some of these crazy things in my next post!

Radial gradients + Mini-path geometry sprites + infinite-scrolling inertial canvas = :)

I’m slowly building up some very cool controls and general learning’s around XAML & SharpDx and I want to share 3 of them with you today..

 

Radial Gradients in XAML

So I’ve said it a couple of times before, WinRT XAML does not have support for RadialGradientBrush, and the Brush class itself is sealed so you can’t even extend it to create your own.

SharpDx to the rescue

SharpDx lets us create RadialGradientBrush for using as a fill

I use that radial gradient as a fill for a rectangle that is rendered on a Direct2D surface in my demo app

 

Here is the result of the Radial Gradient rendered…

 

Mini-Path-Language sprites on Direct2D

I created a simple converter that turns mini path language to Direct2D PathGeometries. It’s not perfect and is based off the open-source library for a Silverlight one “Silverlight string to path geometry”.

I used Blend to create the path asset

Took the “DATA” for the asset (mini-path-language describes the data structure) and converted it to PathGeometries which I used as sprites for rendering on a Direct2D surface.

 

 

 

Infinitely scrolling inertial layer

I created a simple control called an “Inertial Layer” that has sensitivity properties on how sensitive it moves in the X,Y directions. Also these layers are wired up to a “Conductor” control that listens to “ManipulationDelta” data from sensor(touch) events and broadcasts these to the “inertial” layers.

I have 2 new controls “InertialLayer” and “Conductor” that wraps up the functionality needed for infinite inertial scrolling

I have 4 Inertial Layers in XAML and each layer for simplicity sake has a rectangle in it, each layer can actually contain any XAML or DirectX element!

The Conductor is initialized in code and when it starts it pushes data to the InertialLayers ]

 

Video Demos

Here is a video taken from my HTC Titan, excuse the hand movements. I wanted to try to show the smoothness of the animations when encoder isn’t interfering with it.

 

Here is a video taken with Encoder

 

 

Code

 

 

Problems

The “ManipulationDelta” were being negatively impacted by the Direct2D surface high frame-rate rendering. I had to reduce the D2D Surface to 30fps to compensate. I raised the issue here

 

Conclusion

I’m slowly getting the hang of DirectX (via SharpDx) as well as working out how best to combine XAML and DirectX. It’s the combination of these two UI technologies that will create those amazing UI/UX’s that we see in movies.

I have some other very very cool controls and ideas that I’ll show you over time, I’m just slowly working out the kinks in all of them..

Please share your experiences with WinRT, XAML & DirectX. There’s not enough of you guys/gals posting … are you all under NDA and working on super cool secret things ? Hey I am BUT I’m still blogging  

Share your awesome experiences people !!!

Animated gifs in XAML/C# :)

Someone the other day tweeted a link to some very cool animated gifs. I love animated gifs !!

I have always wanted animated gifs in Silverlight/WPF BUT for reasons unknown they never came. They would have been perfect for A LOT of scenarios where creating storyboard animations would have been overkill.

Well turns out WinRT XAML, firmly rooted in Silverlight XAML, also doesn’t have GIF support nor will it probably ever get it. BUT it does have WinRT support and it has access to DirectX & WIC via SharpDx!!

And today on Twitter Jeremiah Morrill tweeted that WIC has support for GIF’s, this intrigued me. Can I build my own support for animated gifs in XAML using DirectX & WIC (via SharpDx) ?!

Challenge is on ……

This is the animated gif I will try to render…

Source : “Savannah Rain” (frommetoyou)

Default XAML/C# metro app project + SharpDx

I created an empty managed metro project and added the necessary SharpDx bits, basically like I have been over the last several posts.

As always this project has a effect renderer that will render to a (Direct2D or Direct3D) surface via a (SwapChainBackgroundPanel or SurfaceImageSource). Again this is what I’ve been doing in all my SharpDx posts to date so just go back to those or the samples if you’ve forgotten

 

Animated GIF resource

Now just add the Animated GIF as a content resource to the project. I could load it in via a URI BUT I thought it would be easier to just include it for now .

 

Load GIF via WIC

Ok this is where it gets interesting, SharpDx gives us access to WIC from C#. Question is does it have the GIF bits.

As I’ve said many times before SharpDx is a very thin/performant wrapper over C++ libraries (but just to clarify there will always be an overhead to using WIC via this wrapper even thou it is small). This wrapper is a metadata mapping generated api so technically the GIF bits of WIC should have come across. Lets test it out.

Lets instantiate a WIC GIFDecoder that will be used to read the animated GIF resource. The “asset” is simple the internal uri to the gif content.

Next when we animate this gif we need to know how many frames to animate. The GIF spec defines this…

Now for each frame we want to get the bitmap of that layer and store it for rendering. There is a lot going on in this code BUT in a nutshell an object is created that lets us get at the bytes of data for a particular frame in the gif. (its easy to follow just step through the code , that’s how I learnt

As it turns out for GIFS the first layer normally has the entire bitmap contents of the GIF and all the other frames above that contain delta information.. Also the ‘delta’ frames are all offseted so we need to get the offset (left,top) of those frames and store them for rendering of that frame later on..

 

So assuming that SharpDx’s GIF WIC works fine that’s basically all we need information wise.. We just need a way to store those frames and render them to the UI.

…. hello SpriteBatch

SpriteBatch

The best way to render a series of textures in the graphics world is via a spritebatch. Lets create a very very dirty one that will serve our purposes ..

I’m not going to go into too much detail of how this dirty spritebatch works, it’s not in the truest sense a spritebatch BUT it will do for now..

This spritebatch will keep a cache of the GIFs frames that I will render on a DirectX Surface. I’ve defined 2 lists to store the static frames ( 0 ) and the rotating frames ( 1 – 7 )

I’ve created a way to add sprites (frames) to my spritebatch via the methods

1. DrawAlways  – sprites that will always get rendered on each loop of the rendering pipeline eg. frame 0 of the aimated gif

2. DrawRotating – sprites that will rotate there appearance with each loop of the rendering pipeline eg. frames 1 to 7 of the animated gif

 

These methods are called when we are decoding our GIF (the section prior to this outlined this process)

The sprite batch, as expected, has the ability to render itself and all it’s sprites. In my example it will render on a D2D surface and will use the much loved BitmapEffect to render the layer .

Also note that the offset(_texturesRotatedOffset) for the layer needs to be rendered for the 1-7 frames

Now if you don’t understand the above just step through the code, it will all make sense..

Does this all work ?!

Does SharpDx’s support for WIC GIF’s work for animated gifs ?!

 

Running the demo

 

Here’s some awesome gifs from a very cool site

 

Here’s the metro xaml/c# demo running one of those animated gifs. Note that the framerate is low because of the simultaneous video recording going on. Without it it runs buttery smooth at 65-70fps.  

 

 

Sample code

 

 

Conclusion

 

I started out this experiment fully expecting it NOT to work… I was so surprised that it actually did . Animated gifs in a XAML managed metro app!!! Damn!!

XAML/WinRT + DirectX/WIC (via SharpDx) is turning into a killer combination of features for building pretty amazing new experiences. I’m only now getting familiar with the frameworks, I can’t wait to see what I’ll be creating a month from now!

I’m loving XAML/DirectX …

Using FX effects in your managed metro apps

In the WPF and Silverlight “desktop” worlds we have the luxury of being able to use shader effects (fx) in our apps, this infrastructure is not available in the “Windows Phone” or “WinRT XAML/C#” worlds.

However thanks to SharpDx and the improved hlsl graphics features in VS11 we can now easily use these same fx effects. Let me show you how I consume fx files in my XAML/C# WinRT apps…

FX to CSO (HLSL)

Our goal is to turn the FX file into something we can use, and if you’ve followed my previous posts the best option we have in DX11.1 world is a custom shader (cso) and thanks to SharpDx we can do that … So lets try to turn this fx file into a cso

Now the HLSL editor and the compile shader features in Visual Studio 11 shader works only in a DirectX/Cx project.

So first step is to create a DirectX/Cx project to host our FX files. In my case I created a Direct3D Application called “D3DFXConverter”

Then copy across the fx files you want to convert into this new project, in my example I’m going to convert a “Pixelate.fx” . I took this fx file from the “wpf pixel shaders libarary” project on codeplex.

Now rename the “fx” extension to “hlsl”

We now get all the editor experience of an hlsl file for our fx file.

What we want to do is try to compile this as a pixelshader, so go to the compiler setting and set the “Shader Model” to “Shader Model 4 Level 9_1 (/4_0_level_9_1)” and “Shader Type” to “Pixel Shader (/ps)” . note that the fx file in WPF is a shader model 2 or 3.

Now if we try to compile the D2DFXConverter you’ll notice some errors

What these errors mean is that the hlsl code is not compliant with the shader model we’ve chosen, we need to make some minor alterations to the hlsl ..

The changes are very minor, I’ve actually converted several fx shaders and in ALL cases the following is all I needed to change

 

1. Wrap constants inside a cbuffer 

 

Before:

After :

 

2. Alter the Sampler Input

 

Before :

After :

 

3. Alter the interface of the main entry point to the shader

 

Before :

 

After :

 

4. Alter the text2D call

 

Before :

After :

 

Now if you compile the project ….

And if you check out the output directory for the successful build you’ll find the binary of the compiled custom shader (cso).

 

We’ve successfully converted our fx to a cso ready for consumption in our managed metro SharpDx app.

 

Consuming the CSO as a custom effect via SharpDx

To speed things up I’m going to reuse the D2DCustomPixelShaderEffect project I created a few posts ago, “Custom Effects – Ripple Effect (Pixel Shader)” .

Copy across the cso into the project …

Create a C# custom effect wrapper around the cso much like what we did in the above mentioned post for the RippleEffect. Basically I just copied the RippleEffect.cs and renamed it to PixelateEffect.cs and changed some of the internals, it was pretty obvious what needed to be changed. If you get confused just re-read the above mentioned post

And the last step is to alter the EffectRenderer.cs to call the newly created custom effect PixelateEffect.cs .

The changes were simple, register the PixelEffect and wire up some gestures to alter the shaders 2 properties “HorizontalPixelCounts”, “VerticalPixelCounts”.

Again I won’t explain how to do that as it was covered in my earlier “custom effects” post.

 

Video of the demo

 

 

The frame rate is low and fluctuates quite a bit because of the video recording going on simultaneously.

 

Demo code

 

 

Conclusion

 

I’ve successfully converted 14 of the FX files from the WPF/SL codeplex project , the ones that weren’t so easy to convert I will cover in future posts.

SharpDx is turning into a great framework that fills a big hole in the XAML/C# stack.

As for the performance of SharpDx, the shader demo’s I’ve been doing consistently achieve frame rates in the 60+ range. It’s definitely performant  …

I look forward to seeing what others create on top of SharpDx!

‘XamlUIPresenter’ , what are you ? :)

There is an interesting API in the WinRT XAML namespace that sounds damn interesting BUT no public examples of how to use it. Even the MS folks are quiet about it ..

What does “XamlUIPresenter” do?

XamlUIPresenter

The documentation explains it as such …

reference : msdn

A xaml visual tree on a D3D surface ?! Seriously that is some awesome sauce right there. I can’t begin to tell you how excited that one sentence makes me

Problem is how do you use it, and is it even working ?!

Blend 5

All Google or Bing searches for the term ‘XamlUIPresenter’ leads back to the msdn page, no examples or demos at all. There’s a conspiracy to hide this api from us, I’m sure of it !!!

Anyway by chance in my normal day to day reflecting of all things that i find interesting, i chanced upon it’s use in Blend 5 for Metro Apps .. !!!!

Let me explain how they use it ….

Blend Design Surface – Artboard

If you don’t already know Blend and Visual Studio share the same design surface rendering engine.

In the world of Expression Blend the design surface is an “DeviceViewArtboard” which is itself an “Artboard” which itself is just a xaml “Control” . Remember Blend is after all a XAML (WPF) app.

A Silverlight App in Blend is rendered on the design surface using the “SilverlightArtboard” which is an “Artboard” (note it doesn’t derive from DeviceViewArtboard)

A WPF App in Blend is rendered through a weird way which I believe is legacy, it hasn’t been re-written with the Artboard approach, if your interested use reflector to see how its done otherwise I won’t cover it here

[use reflector to see how WPF renders to the design surface artboard]

A Metro HTML app in Blend is rendered on a design surface using the “HtmlAartboard” which is “DeviceViewArtboard”

A Metro XAML app in Blend is rendered on a design surface using the “WindowsUIXamlAartboard” which is also a “DeviceViewArtboard”

Simply put the relationship looks like this for the 3 different rendering types of Silverlight apps, HTML Metro apps, XAML Metro apps.

I am only going to concentrate on the XAML Metro app side of things from this point onwards …

an Artboard’s “ImageHost”

Without getting into too much detail each “Artboard” contains an “ImageHost”, which as the name suggests is the host of an image, how that image is created is what interests me

In the case of the WindowsUIXamlArtboard the image host is a class of type “XamlImageHost”

And if you trace it through the actual instance that is assigned to the ImageHost is a class called “WindowsUIXamlImageHost” that is found in the Microsoft.Expression.WindowsXamlPlatform.dll .

I should point out that each of the different development platforms has it’s own “Platform” dll and the dll’s normally follow a common layout & resource pattern.

Everything we want to know is found in the WindowsXamlPlatform …

Now when you look at WindowsUIXamlImageHost in the WindowsXamlPlatform dll there is a presenterWrapper property.

And this PresenterWrapper class happens to sit in the same namespace…

Two very interesting things in this class ….

1. this class references “Windows.UI.Xaml”

2. this class PInvokes the “CreateXamlUIPresenter” call from Windows.UI.Xaml.dll

I think we found ourselves code that uses the XamlUIPresenter

Blend uses XamlUIPresenter ?!

As it turns out Blend, for XAML Metro apps, uses the XamlUIPresenter API in WinRT to render a visual tree on a Direct3D surface which intern is displayed in the artboard in the designer

Not 100% sure about this BUT it appears that the PresenterWrapper.RootVisual contains the xaml visual tree that will be rendered in a D3D surface

The PresentSiteWrapper wraps up all the logic to make the interop calls between managed/native for rendering surfaces handle pointers etc.

Here’s the PresentSiteWrapper interface incase your interested

So it does a bunch of native calls to pass thru the visual tree and return back the D3D surface which in turn is turned into BitmapData for rendering in the artboard.

Conclusion

This XamlUIPresenter api is damn interesting and if it does what i think it does is very very useful. Rendering a Xaml metro app in a “window” in the desktop looks possible from where I’m sitting.

Now that I know how it works and the basic plumbing it needs next step is to try to use it to render Xaml visual trees on a Direct3D surface from within a metro app. I really want to be able to take a visual tree and render to the GPU as is, then possibly print that tree ?!   A future post will explore this!

In Silverlight we can output a visual tree to the WriteableBitmap for printing and doing cool things with, could we do the same with XamlUIPresenter ?!

Maybe this api is intentionally undefined because MS is still working out the story around it. …

A ‘Paper’ like “2-Page” Close Animation

If you follow me on Twitter you’ll know that I love the iPad app ‘Paper’ . And what’s crazy cool about this app is that it was built by ex-Microsoft people that were working on the Courier project

The company is called “53”, and the app can be found in iTunes here .

This app has some really awesomely executed animations and it’s these that I want to try to reproduce in a Metro app.

I’ll try to keep the animations as simple as possible BUT I expect that in the end I may need to resort to DirectX 3D. Also I plan on taking several posts to complete this, as I will be experimenting with a lot of techniques..

Here’s a video of the animations that I hope to reproduce over the course of several posts..

 

“2 pages” to “Closed” view

So this first post will be an attempt to emulate the “2 pages” view to “closed” view. Please excuse the poorly drawn sketch …

Right Page

A simple rectangle half the width of the screen aligned to the right. I gave it an off white fill color just so that it can be distinguished from the left side.

Left Page

A simple rectangle half the width of the screen aligned to the left

 

Slider to emulate 2 finger gesture

Because I wanted to make this as simple as possible I am not going to try to get the 2 finger gesture working right now, in it’s place I’ll have a slider that does pretty much what the 2 finger pinch does .

For the slider I will have it slide 1 to 100 in steps of 1.

The reason we can use a slider to emulate the pinch, is because all we are concerned about with the pinch gesture is the distance between the 2 fingers (index and thumb) over time. So a slider ‘value’ over a min (0) and max (100) will suffice!

 

Animation to “Close” the 2 pages like a book

This is the fun bit, we need to animate the 2 pages in a way that looks like a book closing. I can think of several ways to do this BUT for now the simplest is to use a couple of “Transforms” to fake this 3D effect.

I created a storyboard called “sbClosePages” and arbitrarily am going to animate the page close over 2 seconds. It doesn’t matter how long you choose because in the end it will be the slider, or rather the pinch gesture, that will determine the duration of the close.

Lets start with the RIGHT PAGE :

What you’ll notice is that to get a nice page close effect you can use a combination of RenderTransform (scale, translate) and Projection (RotationY). I achieved this by simply trial and error in blend, sliding figures around to achieve the desired interaction.

The LEFT PAGE is exactly the same BUT the values for the transforms may be opposite as the pages are moving in opposite directions. It really is all about trial and error and visually experiencing the interaction. Blend is awesome for things like this, trying to do this in code or via numbers (no visual) would be very very difficult!

I’m not going to include every snapshot of the left page BUT if you want just look at the demo code to see the actual figures and screens.

What you’ll notice is that for the left page I needed to set the projections ‘CenterOfOrigin’ to achieve my desired effect, this again was a trial and error thing that would be near impossible to do if it wasn’t visual. Blend makes some things so simple that would normally be virtually impossible!

And the reason I needed to set the CenterOfOrigin is because I needed the right side of the ‘LeftPage’ to be the one that stays stationary during the animation.

Hooking up the slider to the storyboard

The last step is to run the storyboard in response to a gesture, in our case it is a slider that mimics a pinch.

First step is to wire up the sliders ValueChanged event.

Basically as we slide we want to set the storyboard to something between 0 and 2 seconds. Remember that the 2 seconds is the duration of the animation we defined above. The slider can move between 0 and 100, so if it moves to 50% then we want the storyboard to move 50% as well, which is 50% of 2 seconds = 1 second.

So the last step is to set the storyboard to the calculated value, and to do that we can use a very cool property on a storyboard called “Seek” . How seek works is it expects the storyboard to already be running, and when we set the seek it will go to that point in the storyboard

So lets kick start the storyboard on the init of the page, note that if we let it run it will just execute the storyboard so we should also pause it immediately!

So now when the slider changes we can “Seek” a particular time on the storyboard

And so when you run the app and slide the slider this is what you get

 

Sample Code

 

 

Conclusion

 

I plan on creating a bunch of posts to replicate a lot of the ‘Paper’ app animations so stay tuned for those. But as you can see it is easy to achieve some of those effects with simple transforms BUT done visually.

Catch you guys around!

Custom Effects – Ripple Effect (Pixel Shader)

DirectX11.1 introduced a bunch of built in effects into Direct2D that span the following categories

In my previous posts I showed how you could use SharpDx to call and render some of these effects inside your XAML Managed Metro apps.

Effect Graphs

Lighting Effects

However in the case where you want to do something that these effects don’t cover, or if you want to create a more optimal effect than custom effects is what you want!

HLSL (a custom pixel shader)

HLSL is the “high level shader language” developed by Microsoft for DirectX for defining their shaders.

DirectX11.1 and Visual Studio 12 introduces a much needed revamped HLSL editing/developing experience. The editor is available in C# & C++ projects, BUT the compilation functionality is only switched on in C++ projects.

Below is a screenshot of what you see when choosing a pixel shader file to add to your C++ project.

Below is a sample of a .HLSL file that is filled in for a simple effect. In this case it’s the algorithm to produce a “Ripple” effect.

In a C++ project you can right click on the .hlsl file and configure how VS compiles that resource. You will see in the screenshot below the yellow highlights the specific outputs for my custom pixel shader

So when this project is compiled this particular resource (Ripple.hlsl) will generate a pixel shader with an extension of .cso (custom shader object)

Remember ALL the above is only possible in a C++ project because that is where the compile logic for shaders exists. I’m hoping, but highly doubt, that these compile features for hlsl arrive in C# projects.

*.CSO in our managed apps

So once we have this compiled custom shader object (cso) we can use SharpDx to instantiate a CustomEffect for it for use in our managed metro app.

So in the sample above take the Ripple.cso and include it as “Content” in your XAML/C# metro app

You’ll notice that we included the Ripple.hlsl file as well, this is a good habbit to get into as it gives you valuable information in understanding how the shader is suppose to work and more importantly how to create a “CUSTOM EFFECT” for the cso.

Our custom effect’s properties

Once we have the cso in our managed project we need to create all the managed wrappers around it to be able to use it. SharpDx makes this process easy …

This ripple.cso takes a bunch of properties as inputs, we are going to put those in a separate file and call it RippleEffects.cs

Now if you refer back to the ripple.hlsl you’ll remember that there was a “Constants” area that defines the effects properties  

Now we know what properties are needed for our custom effect, let’s create a wrapper for our effect and pass these properties into it.

Custom effect wrapper

Now its’ time to load up our cso and pass in our properties and execute this effect! We are going to wrap all this up in a RippleEffect custom effect class!

SharpDx has done all the hard work for us and made doing this easy!

Below you can see we create a RippleEffect class and give it the functionality of a “CustomEffectBase” which is a SharpDx abstraction for a DirectX Direct2D CustomEffect.

 

Next step is to implement the Properties for our effects that we defined earlier. It’s also important to know that we can define default values for our effect properties!

Because we implemented the “CustomEffectBase” we need to override the “Initialize” method,it is in there that we load our cso and assign it to our custom RippleEffect.cs

Also in the Initialize we pass through an “EffectContext”, which is where we are going to load our RippleEffect into. This effect context is important because it has a lot of useful libraries for all things effect related!

I should point out that when we load up our custom effect we need to give it a unique guid “GUID_RipplePixelShader” so that it is unique within the effect context.

Everything else in RippleEffect.cs can easily be understood, just step through the code at your convenience. The important ones I’ve already explained

Using our custom Ripple effect

ok now that we have created our custom “RippleEffect” let’s call it!

In the demos all rendering logic sits in “Render” classes, in our case its called an “EffectRenderer”.

To use this ripple effect I will use it as part of an effect graph. The first step of the effect graph is to create a BitmapSource Effect, which is basically just loading up a bitmap image. We will then pass this bitmap effect into the Ripple Effect creating an effect graph

 

RippleEffect in a SwapChainBackgoundPanel

This is what this effect looks like when we apply it to a SwapChainBackgroundPanel

 

RippleEffect in a SurfaceImageSource

This is what this effect looks like when we apply it to a SurfaceImageSource

 

Code Sample

 

 

Conclusion

Creating custom effects from a cso via SharpDx is dead simple, it’s actually quite fun. SharpDx has made easy to wrap the effect in a managed wrapper which leaves you more valuable time to spend on your actual hlsl

And the other thing I should point out is that SharpDx is a very thin wrapper on Dx and actually mirrors quite a lot of the api’s one for one, which means if you get used to the calling syntax of SharpDx you will feel right at home in the Cx/Dx world!

This post only covered custom “pixel shader” effects, stay tuned as there are other cool effects that I want to cover!!