[Question] Using Shader for rendering static shadows

[The_Next_Guy]

I'm trying to render my scene with shadows to make it look more realistic. I've seen plenty of tutorials on how to render dynamic shadows using an FBO and such but I really only need shadows that are generated once and then kept since my light sources doesn't move. Anyone have any insights on how to do this, that would be much appreciated.

[shatterblast]

I guess it depends on how you want to do it. This process I introduce can work in either GIMP or Krita, but I prefer Krita. In code, you could add a fake 3D effect to "shadows" by treating them like trapezoids or rhombuses by adjusting the top two corner positions as the camera moves or whatever for each object. In LibGDX code, you would want to render the actual object over its "shadow" by rendering the shadow first.

https://www.google.com/search?q=rhombus&oq=rhombus&aqs=chrome.0.69i59.1143j0j1&client=ubuntu&sourceid=chrome&ie=UTF-8

https://www.google.com/search?q=trapezoid&oq=trapezoid&aqs=chrome.0.69i59.2436j0j9&client=ubuntu&sourceid=chrome&ie=UTF-8


Doing the work in 3D would be much simpler over all, but likely, there exists more overhead that way. 2D should have the performance advantage. You could also mix 2D and 3D with Decals I think, but I do not currently know how to take advantage of Decals for the purpose of casting shadows behind them. That particular concept would mimic games like "Don't Starve" and "Diablo 2".

https://github.com/libgdx/libgdx/wiki/Decals


For the purpose of this guide, you should work with a PNG file in RGBA mode. The "A" stands for "Alpha", meaning that you can see the background behind your character or object. With RGB only, you can not see through the picture. It will just act as a solid block of colors.

While the same tools exist in GIMP, I reference Krita's tool-set. The version is 4.1.5.

https://krita.org/en/download/krita-desktop/


1. Make a back-up of your original image and preferably place it in a different folder. Mistakes happen. Also, this version of the image will become the "shadow".

2. Let us work with the back-up. Click on the Image menu at the top of the Krita window. From the pull-down menu Convert Image Color Space. In the dialog window that pops up, Model will show at the top left. We want to set that to Grayscale / Alpha.

3. Click the Contiguous Selection Tool near the bottom of the far left of the Krita window. It looks like a "magic wand", and it sits near the items with dotted lines. Click on your picture, but on somewhere that is NOT your object. It should select the area around your object. Assuming you have a SHIFT key on your physical keyboard, hold that down to select other areas of importance to you. You want the clear parts of your picture selected. If you mess up, just click on an empty area that is not your picture. It should reset.

https://docs.krita.org/en/reference_manual/tools/contiguous_select.html


4. Click on the Select menu at the top of the Krita window. Click on Invert Selection from the pull-down menu. Notice that the dotted lines should no longer on the outside edges of your object. This means your object is ready for coloring to black.

5. Click on the Filter menu at the top of the Krita window. Click on Colors from the pull-down menu. These two items will become your best friend for this guide: Color to Alpha and Color Transfer. For now, we want Color Transfer. Assuming your default color is currently black, the whole thing should give VERY NICE smooth edges with that color. This is why you do not use the Fill Tool like MS Paint has. It will make the edges... well... very edgy. Color Transfer maintains anti-aliasing if your image has it.

6. OPTIONAL: Click on the Filter menu at the top of the Krita window. Click on Colors from the pull-down menu. From that menu, click on Color to Alpha. This will make your shadow transparent. You only need to mess with Threshold at the bottom. Values greater than 100 should make it see-through. Transparency can be adjusted in LibGDX, too, of course.

7. ( CTRL + T ) Now to make the "shadow" behave more like a shadow. You will need the Transform tool. It is located near the middle of the tool bar on the far left of the Krita window. You can also use CTRL + T if you have it available. You then need to click your picture. At this point, I suggest leaving the picture alone. Use Tool Options in the very top right corner of the Krita window. It should be the middle tab. You should see this:

https://docs.krita.org/en/reference_manual/tools/transform.html


8. The main things you need to work with are Scale and Shear. With Scale, you want the "height" at around 20%. I suggest not messing with "width". However, with Shear, you need to make a choice. You only need to mess with the "x" value for that, and "y" can go ignored. However, what if you want to adjust the shadow angles in your LibGDX code for a 3D effect, like in "Don't Starve"? If you change the "x" values for Shear, I imagine it would make that much harder.

9. ( CTRL + S ) Save your file!!!

[The_Next_Guy]

This guide has been very helpful of course but I was looking for a way to generate the shadows in libGDX without external textures. But given your description, I think I know how to do that now. What would I have to put into a Vert program to stretch a quad to 20% height?

[evilentity]

Some engines call it baking. Ie 'bake shadows'. You calculate lighting once and store it in textures. You use them like any other textures your models use. Beats me how to do that easily.

[The_Next_Guy]

I guess its important to say I'm working with a 2D sprite batch rendering tiles and not a 3D environment.

[shatterblast]

This should be enough to at least get you started:

https://github.com/libgdx/libgdx/wiki/Textures,-textureregion-and-spritebatch#spritebatch

https://libgdx.badlogicgames.com/ci/nightlies/docs/api/com/badlogic/gdx/graphics/g2d/SpriteBatch.html#draw-com.badlogic.gdx.graphics.g2d.TextureRegion-float-float-float-float-float-float-float-float-float-