There are few decal systems available for Unity, with development on the main contender ceasing some time ago. This has revived interest among community members who are developing their own solutions, presumably to sell on the asset store. I've been contacted requesting an overview of how we handle decals in Folk Tale. So to help community members develop their own solutions I'm openly sharing my knowledge ( free copy please if someone finds this article useful and launches on asset store! ). You might also be interested in signing up for the Folk Tale Beta.
Shader, Materials and Texture
Let me begin by stating that I have only developed what we needed for Folk Tale. No emphasis has been placed on ease of use, tidying up the Editor UI, or even cleaning up variables, because it was designed to be used only by myself. The system is for baked decals only, and does not include runtime generation which would require a number of additional challenges to be addressed ( such as trimming the decal geometry to prevent stretching, for example generating a bullet decal on the very edge of a steep cliff ).
I settled on using the TransparentShadowReceiver shader available in the wiki. Decals aren't usually shadow casters, so this is appropriate for our needs. I also use a texture atlas. At the moment I have one material per decal type and that gets cloned anyway by each LOD group for alpha fading, but I'm sure this could be optimized to reduce draw calls should you so wish, shifting that work onto a modified shader.
I take a central pivot point, and then raycast each vertex position from above. Then it's simply a matter of constructing the triangles and UVs. In Editor mode this is updated, so you can drag the decal around the terrain as shown in this video until it's in position. The decal is height offset slightly to prevent depth conflict.
The approach isn't perfect, and you'll see towards the end of the video that in some cases the decal geometry can end up intersecting the terrain geometry. There is a solution to that, but we get around that simply by shifting position a little, or using different decal geometry resolutions.
LOD Fading
In Folk Tale our camera can go everywhere, from high altitudes down to the levels you see here. LOD swapping is a major factor in our performance. Unity's LODGroup solution is great, but limited to immediate culling, and has no open API. Before it was released, I'd developed my own solution that supports immediate cull, fade in/out, and scale in/out. I'm currently using iTween, but plan to optimize that by shifting fading into a shader that automatically fades based on distance from camera.
Optimizing Geometry
There are more optimal approaches to constructing geometry if you have information about the surfaces below the decals. In the case of terrain, if you know the terrain resolution, you could reduce the tri count by only constructing what is required. In the screenshot below, the four terrain points would be snapped to the terrain vertex points.
I opted for more flexibility and instead supported an axis vertex count parameter that allows me to modify geometry complexity for most situations.
Static Batching
Finally, I combine decals within a localised area into LOD groups that can be statically batched, since the decals never move. Any commercial decal solution would need to generate decals on the fly (e.g. bullet holes ) so would need to combine meshes on creation to keep the draw calls down.
Enhancements
Many decal systems seem to fall over when the underlying surface has sharp angles, for example in the corner of a room. I've had a few thoughts on how that might be addressed by shifting to cube-based raycasting as shown below. I haven't put this theory into practice though, but others are welcome to give it a shot - let me know how it turns out.
Update
I've migrated the LOD fading from iTween to a set of custom shaders, and it runs much faster, so that is definitely worth doing.
