Thought-to-Video with Stable Diffusion.
It uses fMRI data from the brain to recreate video images.
The process includes two stages:
- Stage A uses Sparse: Coding Masked Brain Modeling (SC-MBM) on a large brain scan dataset.
- Stage B applies a Double: Conditioned Latent Diffusion Model (DC-LDM) to generate images from brain recordings.
More detail:
