Entropy for Kids

Entropy

For kids!

NOOR SIDDIQUI

Credit for the good stuff goes to Tsachy Weissman,
mistakes are mine.

Thanks to Shubham Chandak for reviewing and editing!

This is a story about the adventures of Claude and his older
sister Shannon, and how they played with “entropy”.

What is entropy?

Entropy measures

On Claude’s birthday, Shannon took him to get a balloon.

The first shop they go to has only yellow balloons.

Shannon covers Claude’s eyes, and picks a balloon for him.
Shannon asks, “Claude, what color balloon did I get you?”

Claude can easily guess what color balloon he will get.
They are all yellow!
That’s not very surprising to Claude!

That is “low” entropy.

The second shop has half blue and half green balloons.

Shannon covers Claude’s eyes, and picks a balloon for him.
Shannon asks, “Claude, what color balloon did I get you?”

Claude has a good chance of guessing the right color.
It’s either blue or green.
That’s a little bit surprising to Claude.

That is “medium” entropy.

The third shop has a balloon of every color…
and animal balloons too!

Shannon covers Claude’s eyes, and picks a balloon for him.
Shannon asks, “Claude, what balloon did I get you?”

Claude has a very low chance of guessing the right balloon.
There are so many options!
That’s very surprising to Claude.

That is “high” entropy.

“low” entropy “medium” entropy “high” entropy

Halloween arrives.  
Claude and Shannon scheme together about which neighborhood to
trick or treat in.

Claude and Shannon hear about one neighborhood
where all the parents only give out hershey kisses

If Claude and Shannon go there, they will know exactly what
candy they will get when they ring the doorbell.
That’s not very surprising!

That is “low” entropy.

Claude and Shannon hear about another neighborhood where half the
houses give out Butterfingers and the other half give out Skittles..

If Claude and Shannon go there, they will have some idea what
candy they will get when they ring the doorbell.
That’s a little bit surprising.

That is “medium” entropy.

In the last neighborhood they hear about, every house has a
different treat!

If Claude and Shannon go there, they will have no idea what
candy they will get when they ring the doorbell.
That’s very surprising!

That is “high” entropy.

“low” entropy “medium” entropy “high” entropy

Claude loves hershey kisses. As soon as he heard about the first
neighborhood he grabbed his magician costume and ran for it, leaving Shannon
in the dust deliberating.

Claude went to house after house, filling his pillowcase with hershey kisses.

When Claude got home and reunited with Shannon, he had
plenty of treats, but wanted to play a trick on his parents.
He wanted to convince them he was a REAL magician.

Claude told Shannon, “Let’s do a magic trick where Mom and Dad pick
one of your candies, and hide it behind your pillowcase. Then I’ll
magically know what candy they chose!”

Shannon: “Hmm...so how do we get you to “magically know” what candy
Mom and Dad chose?”

Shannon said, “We need to come up with a way for me to tell you what
candy Mom and Dad picked without talking. A secret code...”

Claude replied, ”Hmm…..I have an idea!”

Claude explained, “We can use my hershey kisses as a secret
language! A wrapped up kiss can mean Skittles! An unwrapped kiss
can mean Butterfingers!”
Shannon replied, ”That’s a great idea! I can drop a kiss in front if
they pick Skittles, and an unwrapped one if they pick Butterfingers”
But what will we do if Mom and Dad choose another candy?

With 1 kiss… we made a secret code for 2 candies

Shannon realized, “We can use another kiss! See, if we use 2
kisses, we can cover twice as many candies as before!”

With 2 kisses… we made a secret code for 4 candies  

Shannon: “If we use 3
kisses, we can cover twice
as many candies as with 2
kisses!”
Claude: “Now let’s try our
trick on Mom and Dad….”

With 3 kisses… we made a secret code for 8 candies

Claude says to his parents, “Pick a candy, any one you want from Shannon’s
bag.”

Shannon holds up the pillowcase to block Claude from seeing, and uses the
kisses code so Claude can figure out what candy their parents chose.

Claude: “I know that you picked an Almond Joy!”
Claude’s parents were shocked!

With 1 kiss… we made a 2=2
secret code for 2 candies  2 * 2 = 4
2*2*2=8
With 2 kisses… we made a
secret code for 4 candies

With 3 kisses… we made a
secret code for 8 candies

 

With 4 kisses… we can make
a secret code for how many
candies? 

With 1 kiss… we made a 2=2
secret code for 2 candies

With 2 kisses… we made a  2 * 2 = 4
secret code for 4 candies

With 3 kisses… we made a 2*2*2=8
secret code for 8 candies

 

With 4 kisses… we can 2 * 2 * 2 * 2 = 16
make a secret code for
16 candies! 

1 kiss 2 kisses 3 kisses

2=2  2 * 2 = 4 2*2*2=8

With every kiss, we double the number of
candies we can make a secret code for.

Another word for a “kiss” is a “bit”.

At school the next day, Claude and Shannon showed off their
trick to all their friends and nobody could figure out how
Claude could guess the candies so perfectly.

Ludwig challenged Claude, “Can you guess what pet I brought for show
and tell in 3 guesses or less? If you do, then you can come with my
family to Disneyland.”

Claude replied, “Challenge accepted!!”

Later, Claude asks Shannon for help. “How can we guess Ludwig’s pet?”

Shannon asks everyone at recess what pets they have, to help
Claude win his challenge.

Shannon finds out that the most popular pet is a dog, then a cat, then
a seahorse, then a hummingbird, and the rarest pet is an iguana.

What pet do you think Claude should guess Ludwig has first?

Shannon explains that she thinks the smartest way to use the
3 guesses is to guess in the order of the most common pets
to the rarest pets.

Guess #1: Is it a…. ?

NO!

Guess #2: Is it a…. ?
?
NO!!

Guess #3: Is it a….

YES!

Claude wins! Thanks to following the strategy Shannon suggested!

Is it a…. pterodactyl ?

NO those are EXTINCT!

Entropy is the average number of questions you need to ask, if you
ask your questions in the smartest way, to get to a correct answer.
The smartest way can mean guessing more likely answers first.

Ludwig has another challenge for Claude. Ludwig picked a number between
1 and 100… and he says Claude has only 3 guesses to find his number.

Can you help Claude come up with the best strategy?

PRACTICE ROUND

Guess #1: Is it ?

NO!

Guess #2: Is it ?

NO!!

Guess #3: Is it a…. ?

NO!!!

This isn’t a good strategy because we lost!
We used up all our guesses and didn’t get Ludwig’s number.

EXPLORED UNEXPLORED

2
3

The problem with this strategy is that we didn’t explore a lot
of the numbers Ludwig could have picked.

The best strategy
gets rid of a lot of

wrong answers
quickly.

Guess #1: Is it smaller than 50?

NO!

Wow! We got rid of 50 wrong answers in just 1 guess!

Guess #2: Is it smaller than 75?

NO!

Nice! We got rid of 75 wrong answers in just 2 guesses!

Guess #3: Is it 87?

YES!

Awesome— Claude won!
If we had used the first strategy… it would have taken 87 guesses to
get the right answer!

What did Claude and Shannon learn?

Entropy measures surprise.

The more surprising an outcome is, the higher
the entropy.

The smallest unit of information is a bit
Also known as a hershey kiss (wrapped or unwrapped)