Page 3 of 5

Posted: Sat Nov 19, 2016 5:59 am
by Deepfake
OcarinaUHD Shader 0.1.0:

I've iterated on the shader build excessively now, it's getting much closer to what I intended. I ended up having to restructure the pipeline and also created additional passes for Ambient Occlusion, Indirect Illumination, Ambient Lighting/Bloom; and also added a few passes for Highpass Sharpen and several Color-related passes. I've added a few passes of a fog/distance coloration shader as well. Some mutes the color saturation in the foreground to address the colors getting blown out due to the light/bloom.

The Highpass Sharpen is actually running with thresholds and is implemented in a way to interact with an inverted cartoon outline shader, what this does in effect is it erodes black values and enlarges ligher shapes without affecting the edges of polygons, only the textures, in such a way as to actually enhance existing high-detail textures on the fly to emphasize thickness of objects and make dark details finer. It also actually adds blacks back into the image, after all of the lighting passes. This is an important step because, without it, the blacks would be washed out and raised up and simply correcting them with contrast and gamma adjustments would actually crush the blacks and create very large black spaces in dark rooms, and distort the saturation/values in the mid-range, due to the game's wide range of environments. Perceptually, it also gives the game a more painted appearance, similar to Nintendo's more recent efforts, but it's more tastefully understated, and it also enhances the Depth of Field blur to give it a more Bokeh appearance without such a high cost.

The Ambient Occlusion now has a secondary, larger, more evenly distributed pass which creates shadows and extra contrast on larger walls. While it supplies local indirect illumination, the bloom actually is being used to spread the colors out, and the screenspace reflections actually create the appearance of further light bounces. Because the bloom is blended with an additive screen function, the color correction shaders are placed later in the pipeline and adjust the gamma levels, highlights, and shadows. This is, in my opinion, a better analogue for light bounces without having to actually create true hardware light bounces. It would not work realistically in 3D, but since I'm working with a 2D image I can take advantage of the screenspace. There is no way for it to simulate a light bounce from anything that is not onscreen, but that is actually somewhat irrelevant to the player. It also has the knock-on that facing a dark wall actually has a similar effect to an HDR shader because the darker the initial screenspace, the less color there is to bloom.


- Before


- After

Posted: Sat Nov 19, 2016 6:16 am
by Deepfake
Detail comparison to show what's happening with the highpass+outline/indirect lighting/etc:


This is all happening through the shader injector, the top image is what the emulator produces and the bottom is still using the exact same textures, etc. This actually is done without any excessive frame loss, although there is a lot of wiggle space in the time-lerp between frames because Zelda64's native framerate is 20 frames per second. Only catch here is that I'm building this at 1600x1200 res instead of 4K, so it will likely have to be optimised much further. Also, GLideN64 and Project64 2.3 seem to have an issue with higher resolutions, likely because some of the functions which could occur in hardware are happening in software instead. Any slowdown this causes at higher resolutions would probably be multiplied. I will test thoroughly with the Mupen64Plus build of GLideN64 eventually, as that is a more stable emulator with native integration for plugins.

Part of what makes the engine render's image so inaccurate isn't just that the colors are not interfacing, but that the N64 is effectively just shading without creating highlights, as it does not create specular highlights. The textures are only ever as bright as they were drawn, for the most part. I have had to make the global image blend actually take highlights into account more, but also mute values which are close to bright.

I've actually been thinking about potentially creating a shader that applies a bright highlight to spaces farthest from concave corners/finds the most convex edges, in an inversion of the depth method that is normally used to apply AO.

Posted: Mon Nov 21, 2016 8:54 am
by Marilink
Much more organic. I like how the line is just a little less distinguishable between the ground and the wall.

Posted: Fri Nov 25, 2016 7:40 am
by Deepfake
[QUOTE="Cornucopilink, post: 1614129, member: 23215"]Much more organic. I like how the line is just a little less distinguishable between the ground and the wall.[/QUOTE]
Yeah, otherwise it kind of looks like it's just floating there. I feel like the space has a lot more depth with the fogging, etc, also.

Posted: Sat Dec 03, 2016 1:37 pm
by Deepfake
A quick update. I've been concentrating on the shaders still, trying to iron out some quirks of performance and editing some of the shader processes themselves to get the game to run in 4k with no frame drops. As of this moment the Reflective Fresnel shader is out, as it's quite intensive and it may take some work to get it playing nicely with the others. Mostly I've been working on the color, and have made a look-up table (LUT) and tweaked the pipeline ordering as well as the DOF, etc, and the AO resolution, and all sample counts, to get the current results. Here's some of the most recent screens. The original screens were 2880 * 2160, the 4:3 aspect equivalent of 4k, time of day is post-dawn.


Detail crop:


I've especially been working on getting the skin gradient to look right, and the models to look soft. The color LUT is doing a lot in that regard.

And here's one at high noon, same shader build;


Both shots are from Project 64 2.3 with GLideN64, vanilla build of the game. Only the main path texture is mine, aside from some of Link's textures such as skin and tunic. The grass detail has been quadrupled in resolution with some pixel scattering used to break up the obviously low detail, although it's still very close to the community HD pack original.

Aside from shaders, I've also put in some work on the actual romhack.

- The Kokiri Forest environmental settings have been changed, including the light colors/levels and fog coloring for all possible times of day. I'm still playing with that some, so no screens.
- Code referring to Link's low-poly models has been altered to call for the high-poly, so the low-poly will no longer been seen in the game.

If you've only ever played the game at native resolution, you may not have noticed that the engine swaps Link's model when he's seen from a distance. There is no obvious reason for the final build of the game to do this save that the high-poly may have been visually indistinct or caused aliasing issues at such a low res. This is what it looks like:


RIP in pieces, awful low-poly Link

Posted: Mon Dec 05, 2016 1:30 am
by 1-up Salesman
This is seriously incredible. I don't get this kind of Ty content on Facebook, I forgot how much I've missed seeing this stuff. Everything here looks so vibrant and dreamy. My current laptop wouldn't be able to run it, but if this gets finished I'll absolutely be trying it out once I have the means.

Posted: Mon Dec 05, 2016 4:20 am
by Apollo the Just
OoT as it really should have been honestly

Posted: Mon Dec 05, 2016 8:45 am
by Marilink
Hey AI, I don't know a whole lot about textures and stuff, but I'm curious. I've heard that N64 textures were basically taking the polygon and stretching out a really low-res PNG on top of it. Are you improving everything this much by simply up-rezzing the images used for the texture overlays?

Posted: Mon Dec 05, 2016 2:30 pm
by Deepfake
^ Eh, well pretty much all texture maps are image files. Some are special images which use color data to store direction data to make a low-res polygon model reflect light as though it had the complex surfaces of a higher-detail model. That's called normal mapping. The N64 could only do a few types of texture mapping, and nothing to be interpreted by the renderer in such an advanced fashion. The textures can store as many channels as Red, Green, Blue, and Alpha. The N64 itself could actually use detail maps also, which are usually finer-detailed maps that would be overlaid atop one with much more broad detail differences. So when you see the grass texture in the field, or the green leaf texture I put in kokiri village, you're actually seeing more than one image blended together.

But anyway, there's obviously a lot going on. I can't change model detail with anything complex like normal maps or parallax occlusion maps, which would actually appear to distort the shape of the models, so model changes will have to be done by actually creating new, higher-detail models like the one I posted earlier. These can be hacked into the ROM itself.

The textures are actually being swapped into the RAM and loaded as the ROM requests them, they're named based on how the default textures are loaded into RAM and the graphics plugin reads the RAM and renders the new textures instead.

Most of the recent improvements in texture you would be seeing are either the community texture pack which mine is being built on top of, or the few that I've changed, or more likely the pixel shader injector I'm configuring for this mod specifically. Pixel shaders are a more modern creation, they actually allow a graphics processor to examine and manipulate individual pixels rather than shading based on vertex method as systems like the N64 did. The short explanation is that pixel shaders can apply extra effects similar to PhotoShop filters, etc, although they can actually read data like the depth buffer and use that to apply the effects based on the actual 3D depth of objects. For instance, in the images above I was running a depth-of-field effect. It looks at the depth, and uses it to determine where to apply blurring. I've setup a similar shader which changes the color based on depth and simulates fog and atmosphere.

There are something like 4 different color grade shaders manipulating the image before and after bloom and ambient light adjustments are made. They affect gamma levels, color saturation and hue, etc. The darker parts of the game will actually change colors based on how bright the environment is, the brightest environments will become very vibrant and look like the high-noon shots above. My texture for Link's tunic is actually darker than default, but because of how the shaders effect that specific color it appears more affected by bright areas.

The shaders are actually handled at the driver stage by Reshade 3.x, Reshade is a shader injector. In the emulator's directory, you place a special OpenGL driver file which reads the Reshade configuration and then inserts shaders when it's rendering the image. I have not written any shaders for Reshade from scratch, although I've modified how a few of them work or am using a few outside of their specification, as it's a fairly complex loadout. Usually Reshade configurations tend to look a bit like an Instagram filter, or they're intended to add light flares or even just extra anti-aliasing to new PC games, you kind of get out of it what your experience can get you, not unlike PhotoShop.

Oh, and so you're aware, uprezzing in a technical sense just means to enlarge an image to a higher resolution, not necessarily adding detail, which wouldn't actually accomplish any better detail but would make the game a heavier load on the emulator.

Posted: Thu Dec 08, 2016 7:21 am
by Deepfake
Working on a new skybox, now. The community pack's was configured with texture offsets for the Rice plugin, but since I'm using the GlideN64 plugin they did not render properly and that caused the grid pattern you can see in previous screens. This is not the final render for the skybox, I am planning to alter the size and depth of its appearance, I just had to get all of the settings more or less accurate to start and this is about as far along as my cloud model has gotten. Rather than just paint it, since the sky is actually rendered on the inside of a cube it needs special projection methods to ensure that the dimensions of the box are not easily visible. So lots of messing around with projection dimensions, more or less.


I'm going to be altering the size of the clouds, brightness of the image, etc. Right now it's a bit dimmer than intended. I also need to put more clouds in the distance so not everything looks local.

The difference in offsets between plugins was a bit of a headache, and on top of that I had no documentation to determine the order of the individual textures (yeah, it's not just one big image or something but tiles). So I labelled all of them and checked out how the game assembles them, like this:


The Rice plugin expects an offset on the bottom and right of the image to the tune of 7 pixels out of 256. GLideN64 distributes these offsets on all sides, with the image centered. Again, no documentation, so I made all tiles with no offsets and one with centered offsets, hoping to see which side it skewed towards, and was pleasantly surprised when the centered offset slotted in just fine. I used some batch processes to apply the same changes and not waste all day applying them individually by hand.

Posted: Fri Dec 09, 2016 4:38 am
by Deepfake
My GPU keeps crashing because the skybox render is too complex. I sent it to my drawing tablet to use its i7 to render slowly. Still faster than the old AMD CPU in my desktop and I at least can still use the desktop while it does its thing. I wouldn't be surprised if the heat were affecting the GPU, but it's nothing but cool in the desktop case so I guess Blender's just asking too much of it at once.

Posted: Sat Dec 10, 2016 5:26 pm
by Deepfake
Update for another texture I finished earlier in the week. It's one of the floor tile textures for the Temple of Time. I've included comparisons with vanilla textures and the community hi-res pack.


Aside from correcting the lighting to shade for depth, I also had to correct the problem of missing grouting gutters of the community HD texture. For some reason, whoever made it only included the grouting between some of the tiles. I've also noticed that while the community pack mostly seems to have gone for a clean marble look, the original Temple of Time textures actually look very worn and unmaintained. I will probably address this and perhaps I will modify my texture in the future once I've done the others.

Posted: Sat Dec 10, 2016 6:15 pm
by Apiary Tazy
Mmm, that's some good tiles there.

Posted: Sat Dec 10, 2016 6:28 pm
by Deepfake
[QUOTE="Festivus Ten, post: 1615699, member: 19345"]Mmm, that's some good tiles there.[/QUOTE]

Posted: Sun Dec 11, 2016 3:18 am
by Apollo the Just
I love.

Posted: Sun Dec 11, 2016 8:35 am
by Deepfake
Thank you all for your continued interest and support, it is very inspiring!

Posted: Sun Dec 11, 2016 2:28 pm
by Random User
It's some impressive work so far, especially dealing with the undoubtedly wonky workarounds Nintendo probably had to use at the time. I'm excited to see and play the finished product!

Posted: Mon Dec 12, 2016 12:54 am
by 1-up Salesman
Am I going to have to sacrifice sixteen albino zebras to be able to run this?

Posted: Mon Dec 12, 2016 3:37 am
by Deepfake
[QUOTE="1-up Salesman, post: 1615819, member: 35098"]Am I going to have to sacrifice sixteen albino zebras to be able to run this?[/QUOTE]
I've got a GTX 970 with 4gb of GDDR5 RAM, I'm running the 32-bit version of Project64 2.3 on a now-older CPU, with limits removed on the emulator so it can access a maximum of 4GB RAM. To run it at 4k/UHD at the game's framerate without drops, you would need that or better. The textures can only occupy a maximum of <4GB RAM in total because they have to be held in RAM beforehand. At the moment, the GPU is bottlenecking the emulator because the shaders are a heavy load, although I'll eventually do a lightweight loadout of the shader and I will probably do a low-end version of the texture pack when it's completely finished for users with less video RAM/slower GPUs who still want to target 1080p.

I'm still fighting with Blender over my cloud render for the skybox. The material shader is too much for it, so the process keeps timing out when it gets to dense geometry. I had to cancel the tablet render because it actually was up to something like 40 more hours and I don't want to stress the CPU with 100% load for that long, so I've been trying to optimize the files and I've decimated the cloud models as far as I can before the hit in geometry would be noticable. Still hanging on a certain part of the model, of course. I guess I'll reduce some of the volumetric passes.

Posted: Mon Dec 12, 2016 11:06 am
by Deepfake
Okay, problem sorted. A few of the models somehow had their normals inverse of what they should have been. Recalculated and now it's off and racing.