Stumbled onto a way to speed up stable diffusion renders by 40%

I was messing with xformers and tried switching to - opt-sage-attention on a whim, and my 512x512 generations dropped from 8 seconds to under 5. Has anyone else found big gains from that flag or is there something even better out there?

2 comments

2 Comments

robinson.matthew14d ago

Whoa, nice find with that flag! I was stuck in the same boat around 7-8 seconds per 512x512 and it was driving me nuts. I gave the - opt-sage-attention a shot after seeing a random reddit comment and yeah, same thing, dropped right down to around 5 seconds. What really pushed it over the edge for me though was also bumping my batch size up to 4, since the attention optimization let me fit bigger batches in my 8GB card without crashing. Now I'm doing like 4 images in around 12 seconds instead of 8 seconds for one. The tradeoff is the images can look a tiny bit different sometimes, but for prototyping prompts it's totally worth it.

jesse98814d agoTop Commenter

Ha! So the secret sauce was just cranking batch size up and praying the card doesn't combust... my 8GB card sounds like a jet engine now when I push it to 4, but hey, 12 seconds for 4 images beats the old 8 seconds for 1 any day. I've noticed the same thing with image quality though - sometimes you get these weird artifacts that look like someone smudged the edges with a thumb. But hey, for testing prompts about "cat wearing a top hat" for the 50th time, who cares if it's slightly wonky. Just glad I'm not waiting half a minute per image anymore, feels like I'm actually making progress instead of watching a loading bar.