Discuss ch 1 - 2.6 of "Elements of Information Theory" [series meeting]

Here's a popular text in Information Theory which goes into some detail on data compression and communication channels. Definitely a good one to read through!
http://www.amazon.com/Elements-Information-Theory-Telecommunications-Processing/dp/0471062596

For the first meet up, we'll get together and go over the first chapter (an introduction/overview) and then get into the fun stuff!  We'll stop after section 2.6 on Jensen's Inequality.  We'll work some problems together, and help everyone understand the material before moving on to the rest of the chapter!  I'm currently working on the venue, and can likely host it at work.

After 5 or more people comment saying they've read the material, we'll set a date and location!

 Recommended Prerequisites:

Probability theory: (moderate level) Joint distributions and marginalization; conditioning; Bayes' Theorem

• Analysis: (beginner to moderate level) Know what a set is; know what a logarithm is; understand the basic ideas of proofs by induction, contradiction, etc.

Join or login to comment.

  • Kirill

    Thank you, Adam, Andrew for hosting a great meetup.

    December 21, 2013

  • Adam K.

    Hey Everyone! That was a good first meeting! Just for reference, we decided:
    1) We'll do a book meeting every other week, and a paper meeting in between (every other week). That keeps up the pace of learning, while not going too fast through the books for people with busier schedules.
    2) We'll pick problems for the book meeting (I'll probably post some recommended ones) and volunteer via the message board to work out/present the problems to the group. If nobody volunteers, I'll have a couple [pre-solved problems :) ] ready to present myself.

    I think it goes without saying that these are just guidelines, and we're always open for some Q&A/group discussion. Problems are a great way to bring out the finer aspects of the material, and we can always break the presentation format to clarify the material.

    December 19, 2013

    • Adam K.

      Thomas: yep! We decided to read the rest of Chapter 2. We're going to meet again the second Wednesday of January (the first is New Year's Day).

      December 19, 2013

    • Adam K.

      Ivan: We didn't compare the two intentionally, but there's a possibility one is a little thicker than the other. We'll have to compare next time. In the meantime, I'll post the subheadings of each chapter in the details for the next meet up, so you can make sure all the content is there.

      December 19, 2013

  • John M.

    RSVP is closed. Can I still show up?

    December 18, 2013

  • A former member
    A former member

    Novice just getting interested in information theory

    December 16, 2013

  • Adam K.

    I've filled out the form to request a room at my work (Buzzfeed) for 6:30 Wednesday the 18th. I'll set the meetup date/time once I've heard confirmation.

    December 9, 2013

    • Adam K.

      noooooo! Ok. hahaha. next time? after the holidays?

      December 12, 2013

    • Mike S.

      I have another Meetup for this date, but that event is on the same block, so maybe I'll be able to stop in for a while.

      December 13, 2013

  • Adam K.

    Hey Everyone! Just wanted to say around this time tomorrow is the cutoff for when I'll be tallying all the preferred meeting days, and reserving the venue! Please be sure to let me know when the week after next is the best day for you to meet!

    December 4, 2013

  • A former member
    A former member

    Just piping in to say that I've done the reading.

    December 1, 2013

    • Adam K.

      (and in the future, we'll definitely choose books more democratically)

      December 1, 2013

    • A former member
      A former member

      I assume that the material in these sections is pretty fundamental and probably found in almost every book on this subject. So I don't think that there would necessarily be much lost if we were to switch. That said, I don't think it's really that important to have the perfect book for the topic anyway...putting effort into any (reasonably chosen) book on the subject is much better than fussing about which one is best. Presumably the discussion of the material would flush out any ideas/opinions that might be covered different in other books. First it's good to get moving in any direction...then you worry about whether you need to turn the wheel.

      2 · December 1, 2013

  • archisman r.

    I read it early in my grad school. The algebraic aspects are well covered. At least, you will get a nodding acquaintance with connections with coding theory, and the relationship between bandwidth, quantization accuracy and so on. The analytical side of the book is weak. I believe(it's been a while) only weak convergence is covered, which leads to much weaker results. This is easily remedied by supplementing with something like Kinchin's book, which is available Dover edition, as well as with Shannon's book/paper, which also seems to be carried by Amazon. One thing I liked about the Cover and Thomas book is the coverage of the typical sequence approach. Application to statistics is also covered, but is a little cursory, but there are definitely other books that can serve for deeper studies.

    November 28, 2013

    • archisman r.

      Kirill, I did not want to suggest it is a bad book at all. I think it is a great first book. The first few chapters are probably all that is needed for this. Adam, I will try to get some of that statistics from information theory out. It has been some time. There is of course, the information geometry stuff also, and that is related, but that easy

      1 · December 1, 2013

    • archisman r.

      Sorry, my typing clearly needs work. I seem to hit enter instead of backspace. Anyway, I will try to fig up the statistics reference.

      December 1, 2013

  • ENGRPOD

    I CAN ONLY GO TO THE DISCUSSION ON INFO THEORY IF IT'S REASONABLY NEARBY. IT WOULD NEED TO BE SOMEWHERE IN NASSAU, OR WESTERN SUFFOLK, NOWHERE IN NYC, THERE, I CANNOT GET TO, UNFORTUNATELY. ONCE WHERE IT WOULD BE MIGHT BE KNOWN, PLEASE TRY TO SAY WHERE, AND WHEN, IF AT ALL POSSIBLE, OBVIOUSLY, MANY THANKS.

    November 27, 2013

    • Adam K.

      Hi Engrpod! It's almost certainly going to be in the city, but I'll keep you posted on whether we'll be able to ustream it or something similar. may be a little better than skype.

      I'm quite familiar with arxiv.org! we'll almost certainly read some things from it from time to time.

      November 28, 2013

  • Serge

    Chapter 1 is titled Introduction and Preview, and it consists of a single section, Preview of The Book, which is 8 pages long.

    November 27, 2013

    • Adam K.

      I suppose that won't give us many problems to solve! My mistake: we should do at least the first part of chapter 2 as well. Let's go up through Jensen's Inequality, sec 2.6 in the second edition.

      November 27, 2013

  • Mike S.

    Is this the first or second edition?

    November 27, 2013

    • Adam K.

      you can compare your table of contents with the link above to make sure you have the same chapters. I'd bet some of the problems are different … we can probably work out a photocopying thing if you need copies.

      November 27, 2013

  • Adam K.

    I have the second edition, but I'd wager they're similar enough. Here's the second edition:
    http://www.amazon.com/Elements-Information-Edition-Telecommunications-Processing/dp/0471241954/ref=pd_sim_sbs_b_1

    November 27, 2013

  • Ivan G.

    November 27, 2013

  • ENGRPOD

    ALSO, IF GETTING TO THE DISCUSSIONS WOULD BE POSSIBLE FOR ME, I DO HAVE SEVERAL OTHER TOPICS OF POSSIBLE INTEREST FOR ALL OF YOU AS WELL, USING CURRENT PAPERS FOUND ON WWW/ARXIV.ORG, IF ANY OF YOU KNOW THAT SITE. JUST ALSO SO ALL OF YOU WOULD KNOW, THERE IS A GROUP THAT GOES BY THE NAME NYC PHYSICS AND MATH AUTODIDACTS ON MEETUP.COM, THAT MEET, I GATHER, AT ONE OF THE MAJOR LIBRARIES IN BROOKLYN. I ONCE TRIED TO LISTEN INTO THEM THROUGH SKYPE, THOUGH THE CONNECTION WAS PRETTY BAD, UNFORTUNATELY. HOWEVER, I ONLY MENTION IT SINCE, GIVEN THE APPARENT INTEREST IN MATERIAL OF COMPARABLE CALIBER TO THEIRS, I THOUGHT ALL OF YOU MIGHT CARE TO POSSIBLY LOOK INTO MAYBE SEEKING TO AFFILIATE WITH THEM. FROM WHAT E-MAILS FROM THEM I'VE GOTTEN, THEY SEEM TO BE PRETTY ENGROSSED IN CLASSICAL MECHANICS, USING THE STANDARD TEXT BY GOLDSTEIN. THEY MAY HAVE OTHER INTERESTS, HOWEVER, THAT IS THE ONE I MAINLY HAVE READ THEY HAVE BEEN GOING INTO OF LATE, AND I GATHER THEY EVIDENTLY SEEM TO BE FAIRLY ACTIVE.

    November 27, 2013

  • Adam K.

    Once 5 (or more) people comment that they've read the first chapter, we'll set a location and date to discuss! We'll also set a schedule for going through the rest of the book.

    November 27, 2013

19 went

People in this
Meetup are also in:

Imagine having a community behind you

Get started Learn more
Henry

I decided to start Reno Motorcycle Riders Group because I wanted to be part of a group of people who enjoyed my passion... I was excited and nervous. Our group has grown by leaps and bounds. I never thought it would be this big.

Henry, started Reno Motorcycle Riders

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy