Close

Sky Fall

A project log for Metaverse Lab

Experiments with Decentralized VR/AR Infrastructure, Neural Networks, and 3D Internet.

alusionalusion 04/06/2015 at 06:240 Comments

I opened up a 360 photosphere image into Gimp and created a new layer from visible, started to mess with the curves and contrast until the sky was blanketed white, and then applied edge detection in order to select and remove the inverse of the mountains and valley -- leaving only the sky. The original picture was not properly exposed but I'll work with what I got.

I exported a copy of the transparent png and opened up Pixel-Drifter, a powerful pixel-sorting tool that is written in Java. Don't try to process the picture right now because the resolution is still set much too high. The first step you want to do is to mogrify the image to 50% and reduce the size of the png file. If your image is still too large, resize it again but keep in mind that its now 25% of the original size and you must resize it 400% when you're done glitching it. The command is part of Imagemagick suite.

mogrify -resize 50% image.png

I opened the image in drift and set the program to auto-save frames every 2%, tweaked the sorting, then I hit Ctrl-T and let it rip. It's going to generate a lot of frames. You can watch in real-time as the sky falls apart. Hit ctrl p anytime to pause, tweak settings, and let it play further if you want to change the way its drifting. Resize back them back now:

mogrify -resize 200% frame*

In order to composite the pixel distorted effects back into the background of our original photograph, we'll use a for loop:

for f in frame*; do convert $f -channel Black -separate +channel original.jpg +swap alpha Off -composite new-$f; done
These files are still huge, so in order to fix that you would first convert them all to jpg:
mogrify -format jpg *.png

From here you can preview using animate command or ffmpeg to combine the frames into a video like so:

ffmpeg -start_number 1 -i out-%3d.jpg -vcodec mpeg4 test.avi
cat *.jpg | ffmpeg -f image2pipe -r 1 -vcodec mjpeg -i - -vcodec libx264 out.mp4

If you want to add audio you just need to include it into the command with -i music.mp3.

Finished: Sky Fall


Now its a matter of taking these newly edited photospheres and either creating a 360 videosphere out of them or putting them inside of a Janus skybox. I don't recommend the skybox option because it took people a long time for them to load the images. The best option might be to just load a different photo every hour but not all at one time. I'm going to experiment with this technique further so keep an eye out for what's down the line ;)

Discussions