AI Safety Gridworlds

I laughed way too hard at the "unplugging itself to plug in the vacuum cleaner" analogy.

"it will volkswagen you" LOL

I SACRIFICE ALL MY HP TO VACUUM THE LAST SPECK OF DUST IN THE HOUSE

There will be another video "if people want"? The people want.

I Love the question at the end on "if we would like to see more". Of course we would. We're not here because we don't want to see more Robert Miles

rob miles & isaac arthur collaboration is nerd heaven

Love the Tron music at the end btw

I love the ukelele cover of Daft Punk going on there. Are the outro songs played by you, Rob?

OMFG you have a channel of your own and I only learn of it today. After many years of longing and begging for another tiny little breadcrumb from Brady I stumble upon a ten-storey cake with a watermelon on top. There goes my night. And my waistline.

I appreciate the green colour cast to this video that makes it seem like you're broadcasting from within The Matrix

Great video as always, especially appreciated the tron legacy reference! Most people don't even seem to remember it exists so seeing your channel reference my favourite movie twice bad been good :)

Regarding the exploration vs exploitation trade-off: I feel you are a bit imprecise with the terms at 5:10 ish. There is a massive difference between knowing that you will have N more trials or having infinite trials. If the number of trials (overall or remaining) is bounded then we can solve this optimally. It might not always be computationally feasible right now, but at least we know how to do it in theory. With infinite trials on the other hand there is no harm in trying a new thing each time as you always have infinitely many trials left to later on exploit your findings. In this case it is not clear how to optimally trade-off exploration vs exploitation.

The Grid. A digital frontier. I tried to picture clusters of information as they moved through the computer. What did they look like? Ships? motorcycles? Were the circuits like freeways? I kept dreaming of a world I thought I'd never see. And then, one day... Edit: Hey rob, nobody else has done a cover of the grid on ukulele. Would love to have an mp3 of that! It sounds great

Thanks for posting these, very interesting as always!

3:38 "That's what the agent really is". That send chills down my spine for some reason.

you should do a Ted talk

What a coincidence, I was just reading the Gridworld paper this morning!

I've missed your videos!

So this video made me realize just how similar goals and restrictions set for A.I. are to things that trigger serotonin/oxytocin and disgust/pain respectively in organic life. The way the A.I. goes straight for reward functions over what you want them to do via setting said functions reminded me a lot of when Scientists wired up a button to a rats brain so that everytime they pressed it they'd orgasm, and they just pressed it all the time and stopped eating and drinking just pressing that button.. That helps put programming a lot more in perspective.. People do self destructive things all the time to trigger serotonin, so it's definitely important that if we are making something and can control what triggers their serotonin then we have to pay attention to what those things are..

Nice touch with the music, very on theme!