New 3d API

[bach]

Hey everyone,

We've been working hard on getting the new 3d API off the ground. First up: probably most of what is in core right now will have to be revamped.

So my suggestion is that instead of improving the core, let's try and focus our time at coming up with a more future-proof version. If you want to follow and contribute to the 3d API there's a new branch for it on github: new3dapi. Of course, nothing is "stable" on the branch - but let's try and keep it clean and compiling.

Here'a run-down on current status:

FBX Converter

Most of my time is currently invested in creating a functional FBX converter that will work independently of Libgdx. The FBX converter is heavily based off the work gameplay3d has done. We've wrapped their classes and added some additional features on top. The output of the FBX converter is a clean, and easy to read JSON format. You can have a look at the specifications here (still WIP): https://docs.google.com/document/d/1nz- ... eU69g/edit

The idea is that the asset workflow will be something like this: modeller -> obj/fbx/dae file -> converter -> json (binary or text) -> libgdx json loader. Because the converter is written in C++ we can distribute an executable without any dependencies.

The fbx converter currently supports:

• Meshes
• Support for Mesh Parts (basically a list of indices affected by a specific material)
  
• Multiple UV channels
• All the standard stuff (normals, various primitive types, etc.)
• Hierarchy
  
  • Nodes can have children
  • Nodes can have meshparts or bones attached to them
  • Nodes contain the material -> meshpart relationship, so each meshpart can have a separate material id.
• Materials
  
  • Basic material types: Phong & Lambert
  • Standard material attributes: Diffuse, Ambient, Emissive, Spec & Shininess
  • Holds a list of textures
  • Textures support UV translation & UV scaling

So, static model requirements are all there. Only big thing missing is Skinning & Animation information.

JsonLoader

This is the loader that we use on the Libgdx side. It currently only reads basic mesh & material information. We're working on getting this to a renderable state and on par with the converter features.

The idea is that the loader takes the json format and produces a StillModel, SkeletonModel or a KeyframedModel from it. The KeyframedModel would be sampled at load time to produce a memory heavier version, but better performing.

Personally, I'd like this loader to become the go-to loader and we deprecate the other loaders in favour of a streamlined asset pipeline via the FBX converter. It'd make the core code lean & easy to maintain. Plus the json format is easy to work with. We obviously want the loader to support both a binary as well as a text version of the format.

What do you guys think?

3d API

As a general note: We're planning on making the new 3d API focused on OpenGL ES 2.0. Moving forward it makes a lot of sense and I'd say that anyone wanting to make a a serious 3d game is going to use shaders anyway. Anyone got any objections?

To make all of the above work with Libgdx we will need to revamp quite a few things in the process. The model class will need to be overhauled to support hierarchy. We also need to come up with a new Material system. Would love to hear your guys' ideas on how we should best design those classes.

Another aspect kalle_h has worked on quite a bit is the Renderer system. The idea is to make something similar in approach to SpriteBatch. Basically you throw any Model class (with assigned materials) at the Renderer and the Renderer will take care of the sorting, and optimising the render pipeline. The Renderer will also include goodies like a Light system.
The idea is that ModelRenderer is an interface that is implemented in a default renderer we supply. However, anyone is free to add their own implementation of the ModelRenderer to suit their needs & opimisations. It'll allow for easy swap-out depending on the needs, without having to rework the entire render code.

There's a lot of work already done towards a default renderer (PrototypeRenderer), but we'll have to go over it again once we've added all the new features.


Finishing thoughts: Let's get a discussion started with ideas & comments on where we want to take this baby. Ultimately it'd be fantastic we have a solution that's as easy to work with as Box2d & SpriteBatch using our ModelRenderer & Bullet.

Cheers,
Bach

[mzechner]

What Bach said The current Model, Renderer and Material stuff needs a major rewrite.

[xoppa]

The new3dapi branch looks great. It makes you think different about a model and mesh. I'm a bit unsure about the actual rendering of the model, because meshparts don't comply with the current mesh implementation.

Can you explain what JsonNode and JsonMeshPartMaterial is meant for? Or perhaps the overall structure.

As for rendering it would say a RenderBatch class with begin(), end() and render(Matrix4 transform, Model model) methods, that does all of the basic things like frustum culling and sorting. And spits out a bunch of RenderObjects (mesh, transform, material, etc.). Then another class (BatchRenderer) can be responsable for the actual rendering. This also takes away the GLES2 dependency as you can have multiple renderers.

BTW: The current Mesh class doesn't have a primitive type property, would it make sense to add it.

[bach]

The new3dapi branch looks great. It makes you think different about a model and mesh. I'm a bit unsure about the actual rendering of the model, because meshparts don't comply with the current mesh implementation.

Well, Mesh doesn't have any problems with MeshParts . MeshParts are essentially just a list of indices rendered with a specific material. However, we do need to re-implement how Model handles meshes for this reason. But we also have to change the structure around to support hierarchical models anyway.

Can you explain what JsonNode and JsonMeshPartMaterial is meant for? Or perhaps the overall structure.

JsonNode is simply a node that can be a bone, a mesh or even just a transform. When thinking about this format, think more in terms of a "scene". The json file in fact reflects a simple scene-graph. There's technically nothing stopping you from exporting an entire level as an FBX file. Now each node can attach a mesh for rendering. But, additionally nodes can render MeshParts with separate materials. Nodes can have child nodes, etc.

The terminology is a little convoluted at the moment. The JsonModel class should probably be called JsonScene. Or something along those lines. Though we really need to think about how much of this we are going to support. Multi-model files add more questions and problems. The overall goal of the fbx-converter is to create a file format that's easy to parse. Doesn't mean we'll use all of it's contents

As for rendering it would say a RenderBatch class with begin(), end() and render(Matrix4 transform, Model model) methods, that does all of the basic things like frustum culling and sorting. And spits out a bunch of RenderObjects (mesh, transform, material, etc.). Then another class (BatchRenderer) can be responsable for the actual rendering. This also takes away the GLES2 dependency as you can have multiple renderers.

The reason I'm wanting to drop GLES2 support is because it'll be impossible to get the same render results for both. So, in favour of the power of shaders it'd make sense to keep it ES 2.0 only.

I like the idea of a "RenderList" generator. Though I'm not sure if there are any practical benefits in having it as a 2 step process. If someone wants to implement their own ES 1.0 renderer I guess they could do that anyway with the ModelRenderer interface. Thinking about it now though .. a lot of the render optimisations will be ES 2.0 specific. I don't think we can find a general solution that will work for both ES 1.0 & ES 2.0.

Bach

[mzechner]

Ya, we should drop GL ES 1.x support.

I like the render list/rendering split, at least my gut tells me its a good idea. That way common tasks all renderers have to do can be put into a single place. The actual renderer can then care for the actual material batching. I'll have to see it in action, e.g. in a prototype implementation, but i think it makes sense.

We should just avoid to overengineer this thing. The first iteration doesn't have to compete with Unity If we can keep the model loading independent of the rendering, we should be fine and have enough room to experiment.

[BurningHand]

I say go with over-engineering... But that's because I am not likely to ever use the 3D stuff. Seems like you all have some good ideas about the best approach so I'll just sit back and enjoy watching it progress from afar.

[bach]

We should just avoid to overengineer this thing. The first iteration doesn't have to compete with Unity If we can keep the model loading independent of the rendering, we should be fine and have enough room to experiment.

I absolutely agree with this. I'd much rather have a compact core similar to SpriteBatch and we can extend on top of it later. So in saying that, maybe we should revisit the idea of hierarchical models and just go back to a simple Model -> SubMeshes structure?

So the structure would end up something like this:

Model

• SubMesh 1
• SubMesh 2
  
  • Mesh Part 1
  • Mesh Part 2

A MeshPart would essentially just hold indices/material info. MeshParts would be optional. Thoughts?

Maybe simple is better in this case
Bach

[bach]

Also, just sharing a little bit of a milestone we've achieved today. Rendering our first fbx in libgdx:



This is an fbx model that went through the converter and was imported incl. materials to libgdx. Sweet!

Bach

[NateS]

Sounds like good stuff. Nice screenshot!

I don't have much input, being a 3D noob for the most part.

Seems odd to have "Json" prefixes on POJOs. Should it matter what loaded them? FWIW, when you do binary there are libgdx classes DataInput/Ouput that have nice varint methods and a faster and smaller way to read/write strings.

[xoppa]

I just pushed my lightweight hierchical entity/component system to github, which is optimized for performance on android and might be helpfull for ideas on the renderer. It contains a RenderBatch and BatchRenderers for GLES10 (not maintained anymore) and GLES20 (including some basic shadow mapping). It also contains BulletPhysics and an OgreSceneLoader (using a small modified assetmanager). The RenderBatch now uses a Renderable (which is an entity component with a model (based on the distance to the camera) and rendering properties for shadows etc.), which was the biggest mistake I made. Thats why I would say using a RenderBatch with an array of meshes and a separate renderer is the way I would go.