November 10, 2008

Misconceptions about how Flash CS4 implements 3D

Emanuele commented on Keith Peter's blog:
"The reality is that there very little 3D code inside the player, at least native code. Whenever you export a movie that uses the new “3D” features, you’ll notice that about 13KB of AS3 code is attached to it. This code is responsible for the pseudo 3D rendering using same old tricks used by earlier 3D renderers. You see degradation in rendering because those vector objects are transformed into cached bitmaps and handled as texture over triangles. Bitmap resampling algorithms inside the player are pretty good, but not good enough in several situations like this one. The Flash IDE, however, does a good job by making completely transparent to the user that this 3D manipulation is not native to the player itself but rather to silently attached code."

(here comes the snark)
How then am I able to paste one line of code into FlashDevelop and get a perspective-correct 3D rotation?

myMovieClip['rotationX'] = 60;

And my SWF isn't 13 KB, it's 1 KB. So apparently Flash Player 10 doesn't need any "pseudo 3D tricks". Sorry to be Mr. Snarky McSnarksalot, but saying "3D manipulation is not native to the player itself" is just asking for it.

So what is that extra code injected by Flash CS4 actually doing? If you have ASV, you don't have to guess; you can just decompile the SWF and take a look. You won't find any secret calls to drawTriangles(), or even the word "triangle" anywhere (sorry Keith).

What you will find are fl.motion classes. Remember those from Flash CS3? I wrote those to implement the Copy Motion as AS3 feature. For CS4, Rebecca Sun updated them to support 3D properties like rotationX, etc. What happens is that when a CS4 motion tween uses 3D, all of the tween properties are controlled with the Animator3D class. This includes the simple ones like x and y. However, if you don't use 3D, the tween is implemented with SWF tags as it has been for a decade.

Why the dramatic change in tween implementation when you use 3D? It boils down to Adobe deciding not to define new SWF tags for 3D features. I'm not going to comment here on the pros and cons of that decision. The practical result, though, is that 3D motion must be scripted, whether by you or by IDE-generated code. And apparently it's still the case that a movie clip is controlled either by the timeline (i.e. with SWF tags) or by ActionScript. I remember being surprised by this in Flash 4, when one tellTarget to _xscale would break a timeline tween permanently.

The Animator3D class may look complex, but it's not doing any rendering per se; it's doing motion. The rendering code is merely a few assignments to:

this._target.transform.matrix3D

There are a lot of Matrix3D manipulations going on. But they are mainly to convert friendly properties like rotationX to coefficients, and to interpolate between matrices. The Animator and Animator3D classes also have to make all scaling and rotation relative to the transformation point, not the origin. This is trickier than you might think.

I hope this helps explain what's going on with that extra 3D code. The moral of this story could also be a nice tagline for ActionScript Viewer:

ASV: Don't guess what's in your SWF. Know it.

25 comments:

Keith Peters said...

Robert, I was skeptical about the "Flash adds ActionScript to your SWF behind the scenes" explanation. But I'd still be willing to bet that triangle drawing with a bitmap fill is used on some level. You wouldn't see it in a decompiler because, like you said, it's player code. But it's quite obvious that the display object is being drawn to / cached as a bitmap and that bitmap is undergoing 3d transformation and being drawn to a 2d surface. And since the triangle engine is built into the player, I don't see why it wouldn't be used.

Robert Penner said...

I agree, that much is obvious.

Keith Peters said...

As far as the other parts go, that's a great explanation. Thanks!

Robert Penner said...

Sadly, I cannot explain the fish hat.

Keith Peters said...

The fish hat needs no explaining

Robert Penner said...

In Soviet Russia, explaining needs no fish hat.

Emanuele Cipolloni said...

The "attached code" discovery was not made by me, I read about it weeks ago on another blog (don't have the link handy) and effectively on Flash CS4 (betas at least), 13Kb of AS3 was attached only when exporting 3D. By the way, Adobe itself never referred to the new features as "3D", they called it something like "2.5D" whatever that means. Certainly makes sense to include that code inside the player itself rather than attach it to every single movie and of course they thought about translating it in C++. However the fact that the vector objects are rendered to bitmaps and then textured over planes/triangles remains; there is no depth sorting or camera handling, difficult to display real 3D objects or world without them. This is also confirmed by the fact that no new tag has been introduced to describe 3D spatial positioning. So: "Flash supports 3D planes" = YES, "Flash support full 3D" = NO

Robert Penner said...

The Flash CS4 Features page says 3D, not 2.5D:

http://www.adobe.com/products/flash/features/?view=topnew

"3D transformation

Animate 2D objects through 3D space with exciting new 3D translation and rotation tools"

Matthew Fabb said...

The link that Emanuele Cipolloni is thinking of this likely this blog post by Trevor McCauley from Adobe:
http://www.senocular.com/?id=2.44

Emanuele Cipolloni said...

Thanks Matt, it was exactly that post!

But since he works for Adobe, he didn't had his finding labelled as "misconceptions", ah life.....

Robert Penner said...

@Emanuele,
I read the senocular post again and I agree with everything there. I agree with some of what you're saying (points Keith already mentioned about triangles, etc.).

With your statement "pseudo 3D rendering using same old tricks used by earlier 3D renderers", I took you to mean engines like Papervision. In that case, I disagree because Papervision (pre-Flash 10) uses skews that are not perspective-correct.

Also, the context of your comment was Keith's post about the resampling of 3D-rotated display object. I assumed you were trying to explain the behavior with your comment. But he's using the Player 3D features directly from ActionScript. So the 13 KB of code has nothing to do with his situation. I interpreted you as saying the 13 KB was doing triangulation and pseudo-3D Papervision-like skewing. I also interpreted you as saying that you can't do 3D manipulation without the 13 KB.

If I misinterpreted you, I'm sorry. But I read your comment many times and tried my best.

Robert Penner said...

Hopefully we can all agree that 3D rendering *is* native to Flash Player, while 3D *animation* is not.
Keith's post was about rendering, not animation, so that's what I was focusing on.

Emanuele Cipolloni said...

Robert, I was talking about rendering too, even if I hardly believe somobyd will put a 3D object in Flash without animating it somehow (so the 13KB will be all there, but that's not the issue).

Any "normal" 3D rendering application produce the final image by sampling primitives at discreet intervals and put pixels on the screen (I'm simplifying but that's how it works in the end), while Flash renders the primitives (the movie clip) to a bitmap first using a fixed projection, so stricly speaking it is still a 2D rendering, then it projects the resulting bitmap onto a 3D transformed plane. That explain artifacts on renderings as Keith reported.

So, I agree to the fact that Flash transform natively 3D planes with texture applied; it is a good start, maybe in Flash Player 11 there would be fully 3D transform applied to any object or the ability to load .3ds objs, who knows, for the moment those important features that are the basis of any other 3D framework, like PV and so on, are simply not there.

Carl Goodman said...

Apologies for posting slightly off-topic, but as a designr in the middle of a project which requires both 2d and 3d interactions I confess I find Adobe's approach to web 3d somewhat mysterious. The reflections on whether the new implementation of 3d in CS4 is actually 3d or'just' perspectve skewing seem to paper over the fact that Adobe actually has a resonably serviceable 3d engine in it's stable, in the form of Shockwave 3d (although obviously this has been fairly well neglected over the past few years). Back in 2000 at Siggraph in New Orleans I counted 52 seperate 3d browser plug-in technologies, some of which surely must have made it through the dot com bomb; why is itthat there is such a paucity of 3d browser solutions nearly a decade later? From a CGI designers perspective, to be homest solutions such as Collada are a wee bit on the lame side, unfinished as they seem to be by Sony. The 3d application industry (a.k.a Autodesk, following their purchase of SoftImage) has moved on phenominally in the past eight years - dos anybody have any ideas as to why web 3d seems not only to have ground to a halt, but in fact gone back a couple of steps? Thanks for your patience.

chris nuuja said...

The FP10 3d effects feature is in no way meant to compete with Papervision/Alternativa3D/Away3D. If you want immersive 3d environments or 3d meshes, you should use those technologies.

If you just want an iPod style 3d cover-flow or a 3d combo box with 3d elements that pop out and rotate as you use it, then Papervision et al. is probably overkill. Simulated reality does not have to be the only use for 3d.

Ryan Henson Creighton said...

It's the same people who keep down the electric car.

Unity seems to be gaining a lot of momentum when it comes to 3D on the web. i hear it being talked about far more than the also-rans (VRML, Shockwave, Virtools et al)

The thing that mystifies me about CS4 3D is that it doesn't do any depth-swapping. You can pull a plane back along the z axis til it's a tiny spec, and it still won't move behind another clip, if it's layered in front of that clip (2 dimensionally-speaking). It's like 3d, sure, as long as every clip is skewed within its own little 3d world.

Try putting 6 planes together in a cube and rotating that cube clip. You'll get some freaky results, with faces appearing behind faces that they should appear in front of. It's all very Escher.

Seems to me to be a giant pain in the ass to implement som proper depth-swapping in the case of a simple cube, let alone a more complex object.

Chris Nuuja said...

Regarding the lack of zdepth management: yes, its a pain. Its not a lot of code, however. You just have to sort your displayObjects by .transform.getRelativeMatrix3D(root).position.z. Something like:

public function ReorderChildren()
{ // where "faces" is an array of { z:Number, child:DisplayObject }
for(var ind:uint = 0; ind < num3dChildren; ind++) {
faces[ind].z = faces[ind].child.transform.getRelativeMatrix3D(root).position.z;
this.removeChild(faces[ind].child);
}
faces.sortOn("z", Array.NUMERIC | Array.DESCENDING);
for (ind = 0; ind < num3dChildren; ind++)
this.addChild(faces[ind].child);
}

Even then, there are cases where the ordering will be incorrect and require fiddling (just as in Papervision et al). Its a 95% solution.

A "z buffer" is the standard way to handle z occlusion. Its what GPUs use. The problem with z-buffering is that it isn't possible to handle alpha transparency correctly in all cases.
When you see alpha transparency in 3d games, the triangles with alpha are being sorted separately using the same alg as Papervision et al, then drawn into the z-buffer in that order. Its a 95% solution, but overall few triangles in games are transparent and where they are, designers/programmers special case work arounds for the limitation.

In Flash, it’s a good bet most DisplayObjects will have some alpha somewhere. Rather than implement something that has bugs we could never fix, we left complete control over ordering to the author. Perhaps a more ideal solution would be a new Graphics like api that did z-buffering of triangles, drawn in the order the commands are made. That model would work well for the programmer, but wouldn’t really be workable for the existing Flash Authoring UI.

Michael Vestergaard said...

Interesting post, thanks :-)

ranjan said...

Nice discussion. :)
Hey guys, check out some secrets on about Flash CS4 [For those who still dnt checked it.]. Check here http://pravinranjan.com/blog/?p=29

Anonymous said...

myMovieClip['rotationX'] = 60;

does this line of code mean that its possible to use the 3d features of flash cs4 in the flex 3 sdk

Anonymous said...

For those interested, Ralph Hauwert made the "Flash Player 10 Z-Sorting Class".
It works very well, and it's fast. I found it through the blog of gotoAndLearn
Here is the link of the post:
http://theflashblog.com/?p=470

Carl Looper said...

Perhaps what Cipolloni means by "pseudo 3D" is anything not done through graphics hardware (ie. via DirectX/OpenGL etc). There is, however, a middle ground between AS3 scripting and compiled graphics hardware instructions - and that is native CPU code. The all important "division by z" operation is the the most expensive operation in the perspective transformation - and having this natively performed (en masse) to otherwise AS3 prepared data provides the cool peformance.

Carl Looper said...

I might add that all 3D frameworks have important code occuring in the CPU rather than the GPU: code responsible for defining 3d models, animations, camera matrices, etc., and it doesn't really matter whether this preparation is done in AS3, Java, C, or C++.

Anonymous said...

I found this site using [url=http://google.com]google.com[/url] And i want to thank you for your work. You have done really very good site. Great work, great site! Thank you!

Sorry for offtopic

George Profenza said...

Hello,

I got to this post looking for some help, that only a few people seem to able to help.

I am trying to do something 'simple' but for some reason it's impossible for me. All I want to do is get the x,y,z position of a 3D transformed MovieClip using jsfl.

Once a clip enters '3D mode', the .x,.y properties in Element aren't helpful. I had a look in the jsfl documentation and I used doc.translate3DSelection() and doc.rotate3DSelection(), but both return nothing, which is annoying. I wish they returned the new position / rotation.

I tried to create a new motion tween and copy the motion to see if I can get anything from MotionClipboard.xml , but the coordinates are relative. Any way I can get the absolute x,y,z position for a clip ?

The timeline has a Copy Properties option, but I can't find any way to access that from jsfl, or where it writes the positions for pasting. I looked PastePropertiesSpecial.xml, maybe it is similar to PastMotionSpecial.xml, but no luck :(

I'm assuming controlling the 3d transform are directly linked to the new stage in CS4, but I'm stuck at the moment.

Any help ? Pretty please ?