Discuss ch 1 - 2.6 of "Elements of Information Theory" [series meeting]

Here's a popular text in Information Theory which goes into some detail on data compression and communication channels. Definitely a good one to read through!
http://www.amazon.com/Elements-Information-Theory-Telecommunications-Processing/dp/0471062596

For the first meet up, we'll get together and go over the first chapter (an introduction/overview) and then get into the fun stuff!  We'll stop after section 2.6 on Jensen's Inequality.  We'll work some problems together, and help everyone understand the material before moving on to the rest of the chapter!  I'm currently working on the venue, and can likely host it at work.

After 5 or more people comment saying they've read the material, we'll set a date and location!

 Recommended Prerequisites:

Probability theory: (moderate level) Joint distributions and marginalization; conditioning; Bayes' Theorem

• Analysis: (beginner to moderate level) Know what a set is; know what a logarithm is; understand the basic ideas of proofs by induction, contradiction, etc.

Join or login to comment.

  • Kirill

    Thank you, Adam, Andrew for hosting a great meetup.

    December 21

  • Adam Kelleher

    Hey Everyone! That was a good first meeting! Just for reference, we decided:
    1) We'll do a book meeting every other week, and a paper meeting in between (every other week). That keeps up the pace of learning, while not going too fast through the books for people with busier schedules.
    2) We'll pick problems for the book meeting (I'll probably post some recommended ones) and volunteer via the message board to work out/present the problems to the group. If nobody volunteers, I'll have a couple [pre-solved problems :) ] ready to present myself.

    I think it goes without saying that these are just guidelines, and we're always open for some Q&A/group discussion. Problems are a great way to bring out the finer aspects of the material, and we can always break the presentation format to clarify the material.

    December 19

    • Adam Kelleher

      Thomas: yep! We decided to read the rest of Chapter 2. We're going to meet again the second Wednesday of January (the first is New Year's Day).

      December 19

    • Adam Kelleher

      Ivan: We didn't compare the two intentionally, but there's a possibility one is a little thicker than the other. We'll have to compare next time. In the meantime, I'll post the subheadings of each chapter in the details for the next meet up, so you can make sure all the content is there.

      December 19

  • John Mangual

    RSVP is closed. Can I still show up?

    December 18

  • Aneesh Devi

    Novice just getting interested in information theory

    December 16

  • Adam Kelleher

    I've filled out the form to request a room at my work (Buzzfeed) for 6:30 Wednesday the 18th. I'll set the meetup date/time once I've heard confirmation.

    December 9

    • Adam Kelleher

      noooooo! Ok. hahaha. next time? after the holidays?

      December 12

    • Mike Selender

      I have another Meetup for this date, but that event is on the same block, so maybe I'll be able to stop in for a while.

      December 13

  • Adam Kelleher

    Hey Everyone! Just wanted to say around this time tomorrow is the cutoff for when I'll be tallying all the preferred meeting days, and reserving the venue! Please be sure to let me know when the week after next is the best day for you to meet!

    December 4

  • Thomas Nyberg

    Just piping in to say that I've done the reading.

    December 1

    • Adam Kelleher

      (and in the future, we'll definitely choose books more democratically)

      December 1

    • Thomas Nyberg

      I assume that the material in these sections is pretty fundamental and probably found in almost every book on this subject. So I don't think that there would necessarily be much lost if we were to switch. That said, I don't think it's really that important to have the perfect book for the topic anyway...putting effort into any (reasonably chosen) book on the subject is much better than fussing about which one is best. Presumably the discussion of the material would flush out any ideas/opinions that might be covered different in other books. First it's good to get moving in any direction...then you worry about whether you need to turn the wheel.

      2 · December 1

  • archisman rudra

    I read it early in my grad school. The algebraic aspects are well covered. At least, you will get a nodding acquaintance with connections with coding theory, and the relationship between bandwidth, quantization accuracy and so on. The analytical side of the book is weak. I believe(it's been a while) only weak convergence is covered, which leads to much weaker results. This is easily remedied by supplementing with something like Kinchin's book, which is available Dover edition, as well as with Shannon's book/paper, which also seems to be carried by Amazon. One thing I liked about the Cover and Thomas book is the coverage of the typical sequence approach. Application to statistics is also covered, but is a little cursory, but there are definitely other books that can serve for deeper studies.

    November 28

    • archisman rudra

      Kirill, I did not want to suggest it is a bad book at all. I think it is a great first book. The first few chapters are probably all that is needed for this. Adam, I will try to get some of that statistics from information theory out. It has been some time. There is of course, the information geometry stuff also, and that is related, but that easy

      1 · December 1

    • archisman rudra

      Sorry, my typing clearly needs work. I seem to hit enter instead of backspace. Anyway, I will try to fig up the statistics reference.

      December 1

  • ENGRPOD

    I CAN ONLY GO TO THE DISCUSSION ON INFO THEORY IF IT'S REASONABLY NEARBY. IT WOULD NEED TO BE SOMEWHERE IN NASSAU, OR WESTERN SUFFOLK, NOWHERE IN NYC, THERE, I CANNOT GET TO, UNFORTUNATELY. ONCE WHERE IT WOULD BE MIGHT BE KNOWN, PLEASE TRY TO SAY WHERE, AND WHEN, IF AT ALL POSSIBLE, OBVIOUSLY, MANY THANKS.

    November 27

    • Adam Kelleher

      Hi Engrpod! It's almost certainly going to be in the city, but I'll keep you posted on whether we'll be able to ustream it or something similar. may be a little better than skype.

      I'm quite familiar with arxiv.org! we'll almost certainly read some things from it from time to time.

      November 28

  • Serge

    Chapter 1 is titled Introduction and Preview, and it consists of a single section, Preview of The Book, which is 8 pages long.

    November 27

    • Adam Kelleher

      I suppose that won't give us many problems to solve! My mistake: we should do at least the first part of chapter 2 as well. Let's go up through Jensen's Inequality, sec 2.6 in the second edition.

      November 27

  • Mike Selender

    Is this the first or second edition?

    November 27

    • Adam Kelleher

      you can compare your table of contents with the link above to make sure you have the same chapters. I'd bet some of the problems are different … we can probably work out a photocopying thing if you need copies.

      November 27

  • Adam Kelleher

    I have the second edition, but I'd wager they're similar enough. Here's the second edition:
    http://www.amazon.com/Elements-...­

    November 27

  • Ivan Grosny

    Hi Adam, here is a nice list of programming books http://cspray.github.io/my.so-a...­

    And here a great list of free books on Algorithms
    http://resrc.io/list/10/list-of...­

    November 27

  • ENGRPOD

    ALSO, IF GETTING TO THE DISCUSSIONS WOULD BE POSSIBLE FOR ME, I DO HAVE SEVERAL OTHER TOPICS OF POSSIBLE INTEREST FOR ALL OF YOU AS WELL, USING CURRENT PAPERS FOUND ON WWW/ARXIV.ORG, IF ANY OF YOU KNOW THAT SITE. JUST ALSO SO ALL OF YOU WOULD KNOW, THERE IS A GROUP THAT GOES BY THE NAME NYC PHYSICS AND MATH AUTODIDACTS ON MEETUP.COM, THAT MEET, I GATHER, AT ONE OF THE MAJOR LIBRARIES IN BROOKLYN. I ONCE TRIED TO LISTEN INTO THEM THROUGH SKYPE, THOUGH THE CONNECTION WAS PRETTY BAD, UNFORTUNATELY. HOWEVER, I ONLY MENTION IT SINCE, GIVEN THE APPARENT INTEREST IN MATERIAL OF COMPARABLE CALIBER TO THEIRS, I THOUGHT ALL OF YOU MIGHT CARE TO POSSIBLY LOOK INTO MAYBE SEEKING TO AFFILIATE WITH THEM. FROM WHAT E-MAILS FROM THEM I'VE GOTTEN, THEY SEEM TO BE PRETTY ENGROSSED IN CLASSICAL MECHANICS, USING THE STANDARD TEXT BY GOLDSTEIN. THEY MAY HAVE OTHER INTERESTS, HOWEVER, THAT IS THE ONE I MAINLY HAVE READ THEY HAVE BEEN GOING INTO OF LATE, AND I GATHER THEY EVIDENTLY SEEM TO BE FAIRLY ACTIVE.

    November 27

  • Adam Kelleher

    Once 5 (or more) people comment that they've read the first chapter, we'll set a location and date to discuss! We'll also set a schedule for going through the rest of the book.

    November 27

People in this
Meetup are also in:

Starting a Meetup Group is the perfect way to share your interests and spark new friendships

Find your people.

Log in

Not registered with us yet?

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy