some questions about shader parameters

[steve3d]

With new ObjectDynamic and it's shader parameters, we can start to port some speedtree shaders into unigine now.

 

But I have some questions about the unigine's shader parameters.

 

In vertex_leaf.shader, we have:

    float4 row_0,row_1,row_2;
    
    #ifdef USE_INSTANCING
        [flatten] if(IN.instance == 0) {
            row_0 = s_transform[0];
            row_1 = s_transform[1];
            row_2 = s_transform[2];
        } else {
            int3 instance = IN.instance * 3 + int3(0,1,2);
            row_0 = s_instances[instance.x];
            row_1 = s_instances[instance.y];
            row_2 = s_instances[instance.z];
        }
    #else
        row_0 = s_transform[0];
        row_1 = s_transform[1];
        row_2 = s_transform[2];
    #endif
    
    float4 position = float4(IN.position.x,IN.position.y,IN.texcoord_1.w,1.0f);
    float4 vertex = float4(dot(row_0,position),dot(row_1,position),dot(row_2,position),1.0f);

from these codes, I can guess that the s_transform is something related to the object's transfrom, but what I don't understand is the row_0/row_1/row_2, from the declaration of s_transform it's just a 3-element array of float4, so that s_transform is not a transform matrix. then what it is? and what is these row_0/row_1/row_2?

 

From speedtree's shader system, speedtree only transformed the leaf cards' position by mulitpile the orientation matrix like this:

    float3 vInstanceOutVector = normalize(cross(in_vInstanceUpVector, in_vInstanceRightVector));
    float3x3 mOrientation = BuildOrientationMatrix(in_vInstanceRightVector, vInstanceOutVector, in_vInstanceUpVector);

    in_vPosition = mul_float3x3_float3(mOrientation, in_vPosition);

the instance up and right are passed in as instance parameter, and after this, just use these to get the correct leaf cards' vertex position

        float3 vLeafCardCorner3 = float3(0.0, -in_vLeafCardCorner.x, in_vLeafCardCorner.y);
        float4 vLeafCardCorner4 = float4(vLeafCardCorner3, 1.0);
        vLeafCardCorner4 = mul_float4x4_float4(g_mCameraFacingMatrix, vLeafCardCorner4);
        in_vPosition += vLeafCardCorner4.xyz;

so I want to do the same in unigine's shader, I can get correct CameraFacingMatrix from speedtree sdk, So I'm using these to get the leaf card's position

    float4 position = float4(IN.position.x,IN.position.y,IN.texcoord_1.w,1.0f);
    #ifdef LEAFS_BILLBOARD
        float4 corner = float4(0, -IN.leaf_card.x, IN.leaf_card.y, 1);
        position.xyz += mul(s_camera_facing, corner).xyz;
        position.xyz += s_camera_direction * IN.leaf_card.z; // offset to prevent z-fighting
    #endif
    
    float4 vertex = float4(dot(row_0,position),dot(row_1,position),dot(row_2,position),1.0f);

but this has a problem, if the tree is not rotated at all, then I can get the leaf card always facing to camera, but once the tree are rotated, the leaf cards are facing randomly, and not to the camera.

 

So I'm asking here, how to get a object instance's transform matrix, or get the dir/right/up vector from unigine's shader system?

 

I'm using this because Speedtree 6.x's leaf card will not always be a rectangular billboards, it can be any shape defined by the leaf texture. and sometime, speedtree's leaf card will be the wrong position and orientation. So I want to port some vertex shader from speedtree.

 

this is the correct facing leaves(leaf cards) and the tree is not rotated.

 

this shows that once the tree is rotated, the leaf cards will facing to wrong direction.

 

 

[steve3d]

ok, then I just figured out about something. in unigine's shader, it seems

float4 vertex = float4(dot(row_0,position),dot(row_1,position),dot(row_2,position),1.0f);

will get the model_view transformed vertex position. but correct porting speedtree's shader need up/right vector or a transform matrix to compensate the rotation and it will also needed by the wind animation. so my question becomes more easy now(maybe), How can I get an object's up/right vector or transform matrix?

[unclebob]

Hi Steve!

 

Sorry for late reply. row_0, row_1, row_2 vectors are forming Modelview matrix and vertex will be in the eye space, so no need to compensate rotation. All you need to do is just to offset your vertex from center of billboard. You can do that with following code:

normal = float3(0.0f,0.0f,length(row_2.xyz));
tangent = -float3(length(row_0.xyz),0.0f,0.0f);
binormal = -float3(0.0f,length(row_1.xyz),0.0f);

vertex.xyz -= tangent * size_x;
vertex.xyz += binormal * size_y;

Where size_x and size_y are billboard sizes. Hope it'll help.

[steve3d]

sorry, this won't help at all.

 

see the following screen capture:

 

this is speedtree's method of making facing leaves(leaf card) they use these codes to make the leaves

        float3 card_corner = mul(s_camera_facing, float4(0, -IN.leaf_card.x, IN.leaf_card.y, 1)).xyz;
        position.xyz = card_corner + s_camera_direction * IN.leaf_card.z;

in these codes, the s_camera_facing and s_camera_direction are calcluated in Speedtree's CView class, I can get the correct billboards leaves like this screenshot. the real problem is above of my code didn't work on rotated trees. rotated trees will need to make card_corner also rotated. so I can get the correct result. and the leaf_card.xy are leaf cards's planar corner position, z is  the offset value to prevent the z-fighting of leaves.

 

 

this is the standard shader, you can see the leaves orientation is totoally wrong and they all placed in wrong direction.

 

in this picture, this is the standard speedtree v6's leaf cards geometry, as you can see, there is no rectangular billboards any more, only this form of leaf cards in speedtree v6, so the leaf_card.xy won't be the billboards size and there is no way to calculated the billboards size because they are not actually a standard billboard with width and height.

 

 

so the only correct way to make the leave cards out of speedtree v6 is from speedtree's shader, that is the first picture.

 

and we can not revert to use speedtree v5, because there is a big problem in speedtree v5, that is it always need a tree contains branch/frond/leaves, lack of any of these geometry type, the speedtree compiler will refuse to compile this tree. and speedtree v6 don't have this restriction.

 

In speedtree v5, the leaf cards are represented by a center position and width/height, but in speedtree v6, there is no way to make the rectangular leaf cards, thus there is no width/height value for leaf cards to use.

 

and in speedtree sdk's documentation:

 

VERTEX_PROPERTY_LEAF_CARD_CORNER: (x, y, z) values, applies to facing leaves only. The two-dimensional position coordinates for the leaf are defined by the (x, y) pair. In the vertex shader, they're rotated to face then camera then added to VERTEX_PROPERTY's (x, y, z) value to take their place in the overall shape of the tree. The (z) value of VERTEX_PROPERTY_LEAF_CARD_CORNER contains an offset value to prevent z-fighting among other leaf geometry lying in the same plane.

 

so my question is , how can I make the tree leaves also works on a rotated tree?

[unclebob]

Hi Steve,

 

Could you give me more details about how leaf cards in speedtree v6 are created? From what I've understood from your post is that each leaf's vertex has its world position (VERTEX_PROPERTY, leaf center?) and additional screen-space parameter (VERTEX_PROPERTY_LEAF_CARD_CORNER) so we can restore leaf shape.

 

Also I didn't find any rotation information so I assume it should be leaf card specific too. We have leaf card animation that calculates angle per each leaf card, you can modify that to add rotation support. The other thing you can do is to 'bake rotation' right in the leaf card geometry in speed tree if that's possible, so when you'll restore your leaf cards using standart mesh_leaf_base material you'll get them with proper angles.

 

Anyway, mesh_leaf_base don't have custom leaf rotation support out of the box. You can see materials/leaf_00 sample and tune 'materials_leaf_base' states to get better understanding of how leaf animation is working in Unigine.

[steve3d]

OK,

in speedtree v6, there is no concept of leaf card actuallly, there is only facing leaves and mesh leaves concept, so facing leaves can be any shape, like the one I posted, but it will never be the rectangular billboards like leaf cards in speedtree v5. so any planar mesh can be facing mesh, and more complex leaves will use mesh leaf.

 

these are vertex property documents about leaves from speedtree sdk:

 

Note: The terms "leaf cards" and "facing leaves", used below, are more or less the same thing: leaf geometry that turns to face the camera (sometimes called "sprites"). While SpeedTree has evolved away from using these for most models, they are still supported. Originally termed "leaf cards" because they were always rectangular in shape, they can now be arbitrarily shaped and were renamed to "facing leaf" geometry.

 

VERTEX_PROPERTY_POSITION: (x, y, z) values, used by all geometry types. For almost all SpeedTree geometry, this holds the standard three-dimensional position coordinates. For facing leaf geometry, it holds the position where the facing leaf card is placed (see VERTEX_PROPERTY_LEAF_CARD_CORNER below).

VERTEX_PROPERTY_LEAF_CARD_CORNER: (x, y, z) values, applies to facing leaves only. The two-dimensional position coordinates for the leaf are defined by the (x, y) pair. In the vertex shader, they're rotated to face then camera then added to VERTEX_PROPERTY's (x, y, z) value to take their place in the overall shape of the tree. The (z) value of VERTEX_PROPERTY_LEAF_CARD_CORNER contains an offset value to prevent z-fighting among other leaf geometry lying in the same plane.

VERTEX_PROPERTY_LEAF_ANCHOR_POINT: (x, y, z) values, used by non-facing leaf geometry only. Defines the pivot point used by the wind algorithm.

 

 

as you can see, for facing leaves type, the leaf_card_corner.xy is acually a offset value to the vertex position of that vertex's position.

for example, if a facing leaf are made by 6 vertics, then each of these vertex will have different VERTEX_PROPERTY_POSITION and VERTEX_PROPERTY_LEAF_CARD_CORNER value.

 

in speedtree sdk , there is a CView class which will use following parameters to calculate shader needed matrix and vectors:

st_bool CView::Set(const Vec3& vCameraPos,

const Mat4x4& mProjection,

const Mat4x4& mModelview,

st_float32 fNearClip,

st_float32 fFarClip);

then in shader part, speedtree will first calculate a rotated matrix, I think, then use the camera_facing_matrix to multiply the corner vector then added this vector to the vertex position, then shader will get the correct rotated ( already facing to camera)'s local vertex position.

 

then it calculate the wind and all other things. and finally multiply with model_view_proj to get the projected vertex position.

 

I also attached the speedtree v6's standard vertex shader and the rar's password are send to you by pm.

 

3c08b10299dmo_do_w913ab_vs.rar

[steve3d]

and here is the two type of trees exported as unigine's mesh.

 

https://dl.dropboxusercontent.com/u/5158177/trees.rar

 

in this scene, there are two type of trees, one is "leaf mesh", equal to geometry leaf in unigine, and another is facing leaves, equal to billboard leaf in unigine.

 

then you can see the wrong orientation of facing leaves, and the wind effect of facing leaves looks so strange.

 

this scene can be loaded in standard unigine binary.

[frustum]

Sorry, can't unrar your trees.rar archive.

[steve3d]

sorry, this rar is compressed by winrar 5, I've just reuploaded a rar 4 version, so please download again.

[steve3d]

any update on this?

[unclebob]

Hi Steve,

 

Sorry for late reply. I had a look at your sample, thanks.

 

Geometry leaves are looking great though I can't see wrong orientation of facing leaves, especially if you turn off wind animation. Do you have a sample or video with tree which have correct orientation of facing leaves so I can see the problem?