AIR vs. Unity3D. Who’s faster? (Update 1)

Update 1: uploaded some sources (look at the end of article).

Hey there!
Yeah, it’s been a while blahblahblah.. To the point! =)

Sometimes I’ll highlight actual files from the archive, attached at the page bottom.

Sometimes I see Flash developers interested in the Unity3D lately, and I – one of them actually)
I work with Unity for a while already and all I can say – it’s fantastic experience! So cool to learn C# language (I hope other Flash devs make wise decision to code in C# as well), to learn new community and people, to meet a lot of new challenges and to look at 3D world from a new point at all!

Many Flash developers are still uncertain they should try Unity and spend their time learning this brand new world though.. And in some cases it’s built on top of the AIR and Unity3D performance differences obscurity. So I’ll unveil portion of this differences in this article to help those Flash developers make a choice (whatever what they choose)!

All examples I’ll compile with AIR 3.6 and Unity3D 4.1 and I’ll try to keep similar functionality and look of these examples to let them compete.
I’ll test builds on the pretty slow Samsung Galaxy Tab 10.1 and make some tests on the desktop as well.

Intro

Okay, let’s start from comparing empty builds.
To measure FPS in unity3D I’ll use hand-made FPSmeter working with GUIText.
In AIR builds I’ll use different FPSmeters, usually in-built to the frameworks I’ll use.
Well, let’s see to the built apks (I built Captive Runtime in AIR and usual release build in Unity3D):

FPS: I see stable 60 on the mobile device (Galaxy Tab 10.1 resolution – 1280×800).
RAM: I can’t measure total ram used in AIR (including runtime) from the AIR itself using standard APIs AFAIR (let me know if I’m wrong here please); I can use Profiler class in Unity3D for this purposes. Yeah, I could use some external profiler for AIR, but since profilers overhead will be different it will decrease overall accuracy, so I’ll skip RAM measurements at all.
Files size:
AIR – 9223 kb
Unity3D – 7406 kb
If I’ll use “Strip Bytecode” Stripping Level in Unity3D, I’ll get 6946 kb build size. Pretty nice, counting we have Unity3D player there!

Code execution performance

Fine, let’s add TextField to the Stage in AIR and OnGUI label in Unity3D to output there our code execution time.
Let’s start from the arrays creation and filling with random numbers.
I’ll use fixed typed Vector in AIR since it’s a fastest standard array there.
And I’ll compare it with standard typed array in C# – it’s a fastest available there too.

AS3 code:

private const VECTOR_LENGTH:int = 100000;
//...
private function callFewTests():void
{
	var timeBegin:int = getTimer();
	for (var i:int = 0; i < 100; i++)
	{
		makeACake();
	}
	outputTextField.text = String(getTimer() - timeBegin) + " ms";
}

private function makeACake():void
{
	var v:Vector.<int> = new Vector.<int>(VECTOR_LENGTH, true);

	for (var i:int = 0; i < VECTOR_LENGTH; i++)
	{
		v&#91;i&#93; = int(Math.random());
	} 
}
&#91;/code&#93;

C# code:

&#91;code language="csharp"&#93; 
private const int ARRAY_LENGTH = 100000;
private float totalTime = 0;

void Start ()
{
 	CallFewTests();
}

void OnGUI()
{
 	if (totalTime > 0)
	{
		GUILayout.Label(totalTime.ToString("0."));
	}
}

private void CallFewTests()
{
	float timeBegin = Time.realtimeSinceStartup;

	for (int i = 0; i < 100; i++)
	{
		MakeACake();
	}

	totalTime = (Time.realtimeSinceStartup - timeBegin)*1000;
}

private void MakeACake()
{
	int&#91;&#93; a = new int&#91;ARRAY_LENGTH&#93;;

	for (int i = 0; i < ARRAY_LENGTH; i++)
	{
		a&#91;i&#93; = (int)Random.value;
	}
}
&#91;/code&#93;

As you can see I have two for loops there. First calls method with second 100 times. Second one fills an array with the 100000 random ints. Built-in random generators used in both AIR and Unity3D.
After APK installation I launched test apps few times and got these average results:

AIR - <strong>1965 ms
Unity3D - <strong>1600</strong> ms

We see almost same results here, Unity3D code runs a bit faster.
Let's try to remove random generation to exclude built-in random generators speed differences from the test. I replaced it with this simple method:

private int GetTotalGeeksCount()
{
	int a = 23;
	int b = 41;
	return a + b;
}

I looked into both AS3 and C# compiled code to ensure compilers didn't optimize these simple expressions. After I uploaded APKs to the Galaxy Tab, this is a result:

AIR - 837 ms
Unity3D - 224 ms

This time we see a pretty big difference here - Unity3D faster than AIR more than 3 times! I think this difference is important counting we measured very simple code usually presents in every game or application.

Try to compare yourself, maybe you'll get another results (I'll be glad to know about them) - look at the files from archive:
Random*, Clean*

Okay, what if we'll use Flash exporter in Unity3D and look how exported SWF can compete with SWF created traditional way in the desktop Flash Player projector?
Let's increase iterations count to 1 000 000, export form Unity3D (for Flash Player 11.4) and recompile AIR app. After launching resulting swfs in FP Projector 11.6 I saw this results:

AIR - 573 ms
Unity3D - 2580 ms (!)

BTW I found a nice bug while exporting SWF from Unity - myFloat.ToString("0"); stops code execution in current method. It's weird exporter can handle pretty big projects, but fails on 1 simple code line haha.

So we can see this code takes much more time in Unity3D exported SWF compared to the usual SWF.
If we started to compare desktop performance, why not to try usual Unity3D desktop builds? I mean WebPlayer and standalone for PC.
Unity3D WebPlayer vs SWF in FP Projector:

AIR - 573 ms (measured in previous test)
Unity3D - 102 ms

Unity3D is faster more than 5 times. Not bad!
Now let's try usual Unity3D standalone (PC) builds:

x86 - 103 ms
x64 - 124 ms

Looks like it's almost same as in WebPlayer, good. One interesting observation - x64 build works a little bit slower than x86 one.

Moving next - let's remove all array-related code from our tests, leaving only pure for loops and methods calls. Results on Galaxy Tab:

AIR - 4181 ms (http://pastebin.com/yNr3aVyc)
Unity3D - 281 ms (http://pastebin.com/btz405XS)

I tried to compile AIR test using ASC2 compiler with hope to see significant speed increasing, but result was 3991 ms, not so much =(

Well, looking at the tests results I can say AS3 code speed is still an Achilles heel of AIR, like as some years ago. I hope this will change in future though.
Unity3D Calculations*, Simple Code*

2D

Render time! Let's start from 2D with simple test - 3000 small rotating quads.
I use Starling for AIR test (yes, I know about other fast 2D frameworks for AIR, this one is most popular and supported by Adobe though).
Also I use 16 bit colorDepth and Starling's built-in FPSmeter.

I made orthographic camera in Unity3D with Solid Color clear flag.
Also I'll use simple hand-made RGB color shader.
And I disabled Use 32-bit Display Buffer, Use 24-bit Depth Buffer in the Player Settings.

eAhNVDy

Building, uploading to device.. Results, FPS:
AIR - 32
Unity3D - 20

We got pretty interesting, but expectable results. Unity3D uses 3 dimensions to manipulate objects and calculate things, Starling uses and optimized for 2D only calculations. This is a one of the reasons why Starling faster in 2D mode now. Did you know Unity3D team is working on adding special 2D mode? 😉
Quads*

Now let's use a texture with alpha on this quads, reduce quads count to 1000 and add some movement.
I got a texture form the Bohan Valera's Starling Benchmark open source app (Hey, Valera! =) ).
And I used RGBA DXT5 texture compression in Unity since my Galaxy built on top of the Tegra 2 and DXT5 is a best one available for Tegra.

cA6t0eU

Building, uploading to device.. Results, FPS:
AIR - 43
Unity3D - 47

Here we see a bit different results comparing to the previous test. I guess Unity3D is faster here because of DXT5 texture compression suitable for Tegra devices only. If I use more universal RGBA32 compression which gives similar to the AIR's texture quality, I have only 30 FPS. From the other side, I have no not-tegra Android device to try other compression methods. Anyway Google Play supports different builds of the app - with different textures compression methods for example.
Texture*

Overall tests results let me say AIR is faster than Unity in 2D only mode now. I hope brand new Unity's 2D mode will let Unity3D to compete with AIR's 2D frameworks more natural way.
BTW we can see here the true power of low-level Stage3D API (all AIR's GPU-accelerated 2D frameworks are built on top of the Stage3D)! 2D frameworks developers can focus on the only features they need, optimizing performance and making frameworks lighter - very flexible approach.

3D

Enough 2D, let's dig into the 3D!
Aaand, we'll start from the simple standard "bunch of cubes" test)
Let's make small 1000 cubes to avoid fillRate issues and use most simple color shader we have.
I'll use Away3D (4.1.0 Alpha src from dev branch) in AIR build for the same reason I used Starling previously.

fskxVfR

Building, uploading to device.. Results, FPS:
AIR - 10
Unity3D - 49

This is a total Unity3D win! I double-checked I have release AIR build. Here is a test source code for you to check if I forgot something: http://pastebin.com/iNzPZTnM
Simple Cubes*

Okay, new is time to make test a bit harder. Let's take a high poly (17k+) 3ds model, add it 10 times, add 1 Directional Light, change shader.
In Unity I used standard not optimized Specular shader. Total triangles on scene: 173840.

gzCy14h

Building, uploading to device.. Results, FPS:
AIR - 27
Unity3D - 51

Expected results here!
Lighting*

PS

What can I say after all this tests? I'd like to say Unity3D combines high performance, flexible and robust IDE, support for wide range of platforms and nice C# programming language. And this is a really powerful combination!
So if you are the Flash (AIR) game developer and you're looking some alternative migrate to, I'd suggest you to try the Unity3D first!
BTW I used not all possible optimizations while creating these test cases to keep all workflow simple, with default approach since my goal was to try both platforms in similar way, with similar code and with only some common optimizations.
Also it was a long time ago when I used Away3D last time so I could miss something important in my code. Let me know if you'll find some critic performance issues in my code - I'll re-write it and update test results then!

I'll be glad to hear from you something about AIR vs Unity3D performance comparison!

Archive with all APKs I tested:
AIRvsUnity3D.7z (38 MB)
It's a standard LZMA2 compressed one (use 7-zip 9 or newer to unpack)

Update 1:
Archive with Unity3D sources:
UnityPerformanceTest.7z (750 KB, LZMA2)
It's a Unity3D 4.1 project. You can find 3 scenes there with 1 test from each article charpter.


All examples were tested on the 1280x800 (landscape) Tegra 2 device (Samsung Galaxy Tab 10.1) running on Android OS 4.0.4. Correct examples execution is not guaranteed on any other devices.

Found a typo? Please, highlight it and press Shift + Enter or click here to inform me!

Share Button

Comments

AIR vs. Unity3D. Who’s faster? (Update 1) — 35 Comments

  1. Pingback: Benchmarking iOS en Android via Unity | Tom @ La Mosca

  2. Pingback: iPhone Apps (Xcode en ObjectiveC) en Unity | Tom @ La Mosca

  3. In Texture example (1000 image rotation), Are you using asc2 compiler with flashdevelop, maybe asc2 will generate SWFs of a large size and framerate lower than asc1, or your code have some problem. I dont believe Unity faster Air in this case. Do you have the source code of Texture Example, please share 😀

    • I download and run Texture.apk on my device (Xperia S, android 4.1.2). With Unity, framerate is 40fps, and Air, Unbelievable, framerate is 58-60fps. I tried decompile your apk file (air), rebuild and install on my device but frame rate is 45-48fps. I tried use ImageBatch, frame is 48-50. This is your project decompiled, please help. I use flashdevelop 4.4.3, air 3.8 (asc2) and starling 1.4

      http://www.mediafire.com/download/jh9nek9niot6d6w/Texture-fixed.rar

      • Your Xperia S is not built on top of Tegra, so textures compression I used is not best suitable for your device (I used best compression for Tegra devices). I bet you could get better performance using more common compression, like PVRTC.

    • Hey, huyhungkun!
      Thanks for your comment, I didn’t used ASC2 compiler since it didn’t exist at the moment I did these tests. I have no sources left unfortunately.

      • You dont understand my question
        your apk is 60fps (Texture – AIR.apk test in xperia s)
        my apk (your apk decompiled, rebuild) is 50fps
        Maybe code or option in my apk have some problem. Iam using png image. I even tried use atf file (ETC1) but framerate no improve, when i use PVRTC or DXT5, image even dont display on screen. Can you give me some solution. Why is your app reach 60fps and my app reach 50fps

        Sorry because my english not very well 😀

          • Thank for your answer but I sure my project is build in release configuration and Packaging a captive runtime. I use 16 bit colorDepth and renderMode “direct”

  4. Very useful article!
    I want to ask you (and other viewers):

    If you must develop a 2D APP, with minigames (Avatar creator, a game like fruit ninja, a quiz game, a drag and drop puzzle game), multi platform (WEB Facebook, iOS and ANDROID), !important with access to GPS data, with access to XML output by an external Hardware device, Facebook Open Graph integration, what you’ll prefer? Flash+AIR+Starling or Unity3D?

    • Thanks for your comment, Kris!
      This is a trade-off and completely depends on your targets, audience and many other stuff. If you’re ok with Unity plugin (facebook app) – go for it and use Unity for your project (and probably you’ll be able to export swf from Unity if your project will be rather simple) on all needed platforms.
      But if you have any problems with Unity3D plugin usage and maximum audience in web application is your goal, I’d suggest to stick with Flash based solution (pure Flash for web and AIR for other platforms), since Flash Player is still way more popular and wide spreaded comparing to Unity plugin (it’s growing fast though!).

      It also depends on your experience and project’s deadline. If you have short term project and have no any experience in Unity – I’d suggest to keep on going using Flash, and vice versa to minimize any time loss on studying new tech.
      Anyway, it’s a lot of fun working with both Flash and Unity (well, a bit more fun in Unity =P) and I hope you’ll choose the best one suitable for you! Have a good luck with your project!

  5. Pingback: Интересное за неделю 24.03.2013 | Максимов Анатолий

  6. Hey,

    Do you have the source code for your tests, I would like to see how you were running your Starling render tests, for example were you running a custom atch renderer for all the quads?

    Thanks.

    • Hey, Adam! Unfortunately I had not enough time to keep all sources of AIR tests. And I didn’t batched quads by hands since they all are moving and rotating. And I didn’t batch by hands anything in Unity3D as well. Anyway such batching should be done on the engine level if possible, like how Unity3D does!

      • Hey, shame there’s no source left. I presume in your test when using the quads it will have been creating a new draw call for each will obviously would be quite slow for something simple.

        Starling has an optional QuadBatch class (http://wiki.starling-framework.org/manual/performance_optimization) for things like this, the reason it is not applied to everything is because it depends on what you are trying to batch.

        Anyway for the 2D, instead of needing 3000 draw calls, you would only need 2. I’m sure that would give you a huge speed boost.

        And I also think it might be worth noting that Starling isn’t an engine but a framework and therefore requires more from the developer.

    • Hi, Halley!
      I’m not going to benchmark it so far, since it’s a completely different level. I should compare it vs. Unity3D Plugins to allow them compete. Have no time for this now, sorry.

  7. It looks like some of the Framework developers are picking up the gauntlet, and have a go at your benchmark.

    Unity3D has only ONE framework for both 2D and 3D. AIR has many. For example, Genome2D is faster than Starling. And I wouldn’t be surprised in another 3D frameworks beats your Away3D framerate.

    • Hey, Daniel!
      Yep, I wrote about this in the article, I know there are a lot of frameworks for AIR, but I picked Adobe supported only, to compare “official” parts. Probably I’ll compare some other Stage3D frameworks with Unity3D in the next blog post!

        • Hey, Jonathan!
          Unfortunately I have no any experience with Cocos2d-X.
          And I’d like to hold on 2D benchmarks of Unity3D until they release 2D mode!

          • I mean please give us comparison,
            “And I’d like to hold on 2D benchmarks of Unity3D until they release 2D mode!”

          • Cmon, man, your benchmarks are one of few with common-sence within.
            I read your multi-threading one, and you made my day – 2 benchmarks without “omg one more stupid test”.

            Please do the 2D test, we will wait for it 🙂

          • Haha, thanks, but now I have no free time at all. I hope I’ll have some in future to run some more benchmarks, with latest frameworks and tools versions.

  8. Amazing article, thank you! It shed some light on such important subject as performance between AIR and Unity.

    It’s really helpful to know how they perform under different circumstances.

    • Hey, Jackson!) Thanks for your comment.
      I didn’t profile them yet – have a really small amount of time for this tests.
      I’m going to try Scout on some AIR tests soon to try figure out some pitfalls out there.
      I know how you like to dive deep into the such problems though, so I’ll be glad to know any of your findings if you’ll have time for this!)

      System.privateMemory doesn’t return total memory usage for me – it shows 12 megs for a 50 MB process (as I sniffed in the process manager) both on desktop and mobile! System.totalMemory returns even smaller numbers (