In case anyone is interested in a continuation of the question about compound indexes on parallel arrays from last weeks Scaling FourSquare meetup...
Aryeh and I e-mailed back and forth about this a little.. the main problem with converting an array into a decimal representation that you then do bit arithmetic on using a $where filter to determine set membership is that... mongoDB needs to scan over every document in the collection to then do its bit arithmetic before it can know if a document matches.. ?so that query will not benefit from an index. (and it will be slow against a large collection)
I did some testing and posted results to the mongodb-user group:
The first half of the post covers?weirdness?with $all against indexed arrays (it seems way slower than it "should" be).
The second half covers a novel approach to pre-computing set membership values (in your front-end code).. and then using those values to create a query using $in against the decimal/bit representation of your array. ?This is much faster than anything you can do with $all against the actual array.
Granted.. you couldn't use the technique against a single array property that took values from a 40 item set (since you can't have enough RAM to pull that off), but you could create two?separate?decimal/bit representations of the array.. each representing the set membership for half of the potential array items.. and create a compound index on both properties. ?(This sentence probably makes zero sense in the context of this e-mail, but, for those interested, I think it should make sense after reading the mongodb-user group post.)