︎

Hello 👋️ I’m Jessica,
NYC based Product Designer and Creative Technologist

I solve problems through product building and management with my interdisciplinary and transdisciplinary practice which involves computing, design, art, and business. 

👾 tech         💭 design                🎮 media             💸 business


Select Projects

I bridge the gap between product and experience design.

My work is at the intersection of emering media and purposeful applications. 


PRODUCT SERVICE DESIGN ⎤
Design + Tech ⎤
Management ⎤
Machine Learning Stunning Afterglows
FEB-MAY 2021
Image Machine Learning training using StyleGan model. Python coding & Tensorflow
Filed under:
ML, Design, Coding

Overview

I’m working with my own photos of “sunsets” which I’ve taken over the years. These photos have been taken in several countries and regions in the world. Sunset time is special to me and it gives me hope of a day gone and a new day coming. It’s also very dreamy and romantic to see how the sky’s colors change. Most of the sunsets in NYC are beautiful and the colors are stunning especially when it turns pink but this is only because of pollution. The more the air pollution, the more the sky turns pink when the sun is setting.

I hope to use the machine learning from the colors of the sky at sunset time to show variations of patterns in relation to pollution, and also use these patterns to predict more beautiful sunset skies.

Concept Exploration

  • Air index quality affects afterglow colors in sunsets especially in populated and dense areas, where air pollution is higher.
  • Comparing and contrasting between lesser density areas and afterglow colors.
  • Leveraging this tool for creating unique stunning sunset backdrops for design.

Dataset


1st dataset: 1470 photos, 629 frames extracted from 2 videos using opencv [by Aarati Akkapeddi https://colab.research.google.com/drive/1WWHNG4YqGSHfIYIUrC2tmoPQ3HOgU--e]

I have 2 videos which I transformed to images to add to the dataset. In total I planned to have around 2000 images for lengthy periods of training.

This dataset included photos taken in the U.S (several states), Lebanon, Spain, France, Turkey

Resized to 256x256

All in jpg format



2nd dataset: 787 frames, taken in New York City on June 25, 2019

extracted from 2 videos using opencv [by Aarati Akkapeddi]

Resized to 256x256

All in jpg format


Training Process


  • I trained dataset 1 for 20 hours
  • Downloaded pkl files
  • Interpolation
  • Generated a new dataset with generate.py code and selected names it dataset 2
  • I started re-training on the same worksheet, trained dataset 2 for 16 hours
  • Interpolation
  • Then used the last pickle file from dataset 1 training (000266.pkl) to train it with dataset 2 images. (cross pollination). Used the learning from the last dataset onto the latest training.
  • Interpolation
  • Downloaded pckl file 000048 from the last combined training (14th).








Outcome 


Seeds from training dataset 1



Seeds from training dataset 2



Noticeable Outcomes

  • The Fine Styles — Color Scheme is preserved in outputs 1, 2, 3
  • There's adaptive normalization in 1 & 2
  • Comparison between the sunset output in Lebanon (mostly) and NY. Less dense areas were definitely lighter in sunset colors. While pink and orange were mostly in new york sunsets since the air is more polluted.
  • StyleGAN has generated realistic images
  • Slight loss in style and content in training 3


Further Development


  • I WANT TO INTERPOLATE USING DIFFERENT INPUTS AND MANIPULATE OUTPUTS USING EXISTING LEARNING TOOLS
  • EXPERIMENT INTERACTIVITY THROUGH PROCESSING - P5.JS
  • I AM DESIGNING SUNSETS, EACH DESIGN IS UNIQUE AND YOU CAN OWN IT

Applications to look into
  • Image Classification

Upload a picture, and the model tries to classify it depending on what it “sees” on the picture. This model uses transfer learning and is based on MobilenetV2.
  • Style transfer

Blending image style adding these sunset images as backdrops. Using image style to create a sunset.
  • Learning and experimenting with deep-learning challenges and approaches for sky images

First images sample


Process

I have a primary 870 images dataset. I began training with a portion of this set and hope to add all of the images to it.

I used the styleGAN training code and ran into a couple of errors in the first 3 trials after I finally was able to get an output.

The code is still running for a better outcome and I’ll upload more outputs in the upcoming week.


Early stage outcome 



We can notice the clarity of the images becoming better with time & getting closer from looking real.



The below image looks interesting. Since most of the time the sky is positioned upwards and it seems like there’s a typical way we see sunsets, yet here it’s kind of rotated. I’m searching for more interesting findings; in general the colors and shapes seem close to the reality of true sunsets.


Bibliography

A. Implementation and Concept Research


I- What dust and pollution don't do

It is often written that natural and manmade dust and pollution cause colorful sunrises and sunsets. Indeed, the brilliant twilight "afterglows" that follow major volcanic eruptions owe their existence to the ejection of small particles high into the atmosphere (more will be said on this a bit later). If, however, it were strictly true that low-level dust and haze were responsible for brilliant sunsets, cities such as New York, Los Angeles, London, and Mexico City would be celebrated for their twilight hues. The truth is is that tropospheric aerosols --- when present in abundance in the lower atmosphere as they often are over urban and continental areas --- do not enhance sky colors --- they subdue them. Clean air is, in fact, the main ingredient common to brightly colored sunrises and sunsets.


Afterglow colors are affected by smoke, air quality, and other factors. The stunning hues when affected by smoke are mostly pinkish

Source: https://www.spc.noaa.gov/publications/corfidi/sunset/#:~:text=Typical%20pollution%20droplets%20such%20as,are%20on%20the%20order%20of%20.&text=Similarly%2C%20the%20vibrant%20oranges%20and,more%20than%20soften%20sky%20colors.

II- Machine Learning which sunsets are considered beautiful from social media data


Source:
  1. https://twitter.com/Senor_Sunset
  2. Luminar: Sky replacement tool - AI Powered Tool - picture editor
  3. Adobe photoshop https://www.theverge.com/2020/9/21/21449124/photoshop-sky-replacement-tool-ai-machine-learning
  4. SunsetWx - Sunset and sunrise forecasts. Sunset & Sunrise Predictions: Model using an in-depth algorithm comprised of meteorological factors. https://sunsetwx.com/
  5. https://twitter.com/sunset_wx?ref_src=twsrc%5Etfw%7Ctwcamp%5Eembeddedtimeline%7Ctwterm%5Eprofile%3Asunset_wx&ref_url=https%3A%2F%2Fsunsetwx.com%2F
  6. Sunsets, Fraternities, and Deep Learning http://obsessionwithregression.blogspot.com/2016/05/sunsets-fraternities-and-deep-learning.html

Image-to-Image Translation with Conditional Adversarial Networks https://arxiv.org/abs/1611.07004


B. Air Quality Index - Pollution Levels NYC


Air quality index in NYC 5 and ozone at current concentrations in New York City. Health Department estimates show that each year, PM2. 5 pollution in New York City causes more than 3,000 deaths, 2,000 hospital admissions for lung and heart conditions, and approximately 6,000 emergency department visits for asthma in children and adults.



Get in touch for more info & resources on this project

©2022 Jessica Jabr –  All Rights Reserved
Copyright of Original Works © 2022 Jessica Jabr

  Spotlight                          ︎ ︎ ︎ ︎︎