StackOverflow Overflow

StackOverflow is branching out heavily in the past months. First, SystemFault, a sysadmin-targeted site. Then, they purchased SuperUser.com (coming soon) – a site for home users (questions like ‘how to configure my router’).

And now, a meta discussion forum. It seems the process of cloning or ‘flavoring’ SO is now almost streamlined. I wonder how many stack overflows will we have a year from now. In two years? I think it’s rather unfair that only us programmers (and now sysadmins) have such a great resource. What do you think – will SO clone into Q&A forums for other non-computer topics (a suggested name for a law-based SO – ObjectionDenied :) )

P.S.

Apparently,  there’s already a  release date for the platform! Prepare for the onsalught of StackOverflows.

Unsolved Problems in Technology / Computers

In 1900, the great mathematician David Hilbert presented twenty-three open problems in mathematics. A Google search for open problems in computing gives this Wikipedia article with problems like P = NP and existence of one-way functions. I’d like to list a few, more practical, open problems in computing, that I believe will start to get solved in the following years (most are being work upon for years, but I believe the rate of solving them is accelerating).

Here are my choice problems – I am sure I left many important problems out, and perhaps some of the ones I present are more solved than I know. Please, comment if you’re interested.

  • Search – Google has redefined the field in the last ten years, but I think it’s too early to call search a solved problems (otherwise startups like Delver wouldn’t get started :) ). The bleeding edge seems to be in Social/Personal search.
  • Instant video & audio playback – play any video or audio clip, episode or movie. High quality. Free! We’re very close to the solution here. Youtube, bittorrent, Internet radios (I’ve tried last.fm, blip.fm, Jango). We’re not there yet though, all of these still have problems in pricing, availability or quality.
  • Naive AI – the most undefined term in computer history. I’m not talking about Terminator-style AI (yet). What I want is solid, prevalent voice-recognition. Image and video recognition (auto-tagging on Facebook, for starters). Some system I could speak to and say “please find my next open slot on Tuesday in my calendar, invite Oren & Avishay for a design review, oh, and send them the presentation beforehand”. Something that has just enough smarts to automagically integrate all the other smart systems we already have today.
  • Organization. No need to backup your computer, or put everything into folders, or manually tag stuff. No need to remember passwords, logins, websites. Wait, did I upload that image to Flickr or Facebook? Do I have Anton’s contact from Facebook or Gmail? I think I the copy of the seminar I saved to my desktop is more recent than the one on my laptop… The login/password problem, at least, is nearing its solution – usage of OpenID is accelerating, and it is getting more comfrtable to use as well.
  • Availability. Sure, we’ve got iPhone 3g. But still costs 500-600$. And is suffering much criticism. I want a cheap, good endpoint to the worldnet (imaginary word I just invented to describe the combination of all the world’s networks – internet, phone, TV, …). I want one for my office, one for my bedroom, one above the kitchen table, one in the grocery store… You get my drift.
  • Power consumption. Why are laptop battery life measured in hours, not days or weeks?
  • Viruses and other malware. It’s not as prevalent as a few years ago (for me), but I still run across virus-infected files.
  • Universal data portability. Standards, standards, standards, starting from how to write web pages, to how to write software that doesn’t require root permissions to work, how to access address books and social networks. Lots of work in process here, not yet wildly adopted.
  • Information consumption. How to get really good personalized content without being blown away by the quantity or poor quality.
  • SPAM – I’m not sure this problem will go away, it might just get worse. Computers are getting better and better at pretending to be humans.
  • Planning – computers are excellent planners, right? So why can’t I use the computer to automatically plan a trip from Tel Aviv to Kiryat Bialik, where my parents live? I take the train to Kiryat Motskin, and a bus or taxi to Kiryat Bialik, but no single system (as of today, in Israel) is able to plan a complete route using both means of transportation. This bullet is deeply related (some would say included) in data portability, a few paragraphs above.

How about tigers?

Here is a short story written by Aya & myself (Hebrew):

“מה לגבי נמרים?”, שאלתי.
האישה הגיעה למסקנה שאנחנו צריכים עוד חיות בבית, ואני ניסיתי להעלות חיות אקזוטיות מדי כדי שהיא תרד מהנושא.
“לא רון, אי אפשר נמרים, כבר חשבתי על זה. אנחנו נצטרך ארכיטקט מומחה כדי שיבנה לנו אגף מיוחד לנמרים. הם יוכלו להסתובב בבית רק בערב. אולי נקים מושבת רבייה?”
“כמה כסף את חושבת שאני מרוויח בעבודה?”
“חמוס, אולי ניקח חמוס? בעצם גידלתי כבר חמוסים אז זה בעייתי במקצת…”, המשיכה איה לעבור על רשימת החיות בטקסונומיה של לינאוס.
“טוב, אז אולי היפופוטם?”, המשכתי בטקטיקת ההסחה.
“נו רון תהיה רציני, אין לנו בריכה בבית.”
“את בטוחה שהיפופוטמים חייבים בריכה? מה אם נפעיל מזגן?”
“רון הם חיים במים, אל תהיה מגוחך. עכשיו אני רוצה תוכי. בעצם תוכים, נקח אותם זוגות זוגות.”

יופי, עכשיו היא החליטה לפתוח פה תיבת נוח.

“לא מספיקים לך שני החתולים והארנבת שלנו?”, שאלתי בתקווה?
“זה לא מספיק, שני החתולים כמעט לא עושים צרות לאחרונה. הם שקטים מדי.”

התגעגתי לימים שבהם היינו שנינו מאוחדים כנגד החתולים, ויכולנו להתלונן עליהם באושר.

“רון, אני רוצה לראות התנהגויות של עוד חיות.”
“איה, יש לך גן זאולוגי שלם מטר מהעבודה. רק אתמול שיחקת שם עם גור זאבים!”
“אלה לא חיות שלי, זה לא נחשב. חוצמזה, למיכאל ולחברה שלו יש יותר חיות מלנו, יש להם אפילו צמח טורף! נכון, זה לא חיה, אבל הוא אוכל.”
“איה, אנחנו לא בתחרות גידול חיות. לא, זאת לא היתה יכולה להיות תוכנית ריאלטי טובה.”

“בוא ניקח כלב בשביל הבריאות שלך, רק בשביל הבריאות. אתה יודע שהרופאה אמרה שאתה צריך ללכת יותר… אני חושבת שאם בחורה באה ומבקשת שלושה כלבים, זאת לא בקשה כזאת מוגזמת. או אולי עכברים, זה נשמע די פשוט לגדל עכברים.”

“איה, יש לנו זוג חתולים רצחניים בבית. לא חבל לקחת עכברים רק כדי לכלוא אותם בחדר השני, ולגלות אחרי חודש שהחתולים למדו איך לפתוח דלתות הזזה?”

“מה דעתך על דגו? לא גידלתי עדיין דגו.”

בנקודה הזאת בהיתי בה וגירדתי בראשי. על מה האישה מדברת, למען השם?

“היינו בחנות החיות רון, מישל הסביר לך מה זה דגו. או שבזמן הזה היית עסוק בלשחק עם התוכים? אתה יודע מה, מה לגבי התוכים באמת? אתה אוהב לשחק עם תוכים!”

 רון הסכים שדגים זו אופציה סבירה, שאלתי אותו מה לגבי אקווריום הייטק. על פי המבט על הפנים שלו ראיתי שהוא חושב על גידול מתכנתים זעירים בתוך אקווריום.

“זהו”, חשבתי לעצמי ” נתחיל לגדל מתכנתים… איה תתעשתי”, אמרתי לעצמי. “את יודעת שרון רוצה חיות עם תחזוקה נמוכה ומתכנתים דורשים אוכל מיוחד וציוד אלקטרוני.”

“אקווריום הייטק” חלקתי ממשנתי עם רון “זה פשוט אקווריום יקר מאוד עם הרבה ציוד, שאפשר לגדל בו מינים של דגים שדורשים תנאים מיוחדים.”

רון עדין נתקע על עניין הדגו, התחלתי לחשוד שהוא לא רציני מספיק, הצחוק המטורף שלו היה אחד הסימנים. פסלתי את האפשרות, ברור שרון רוצה עוד חיות. אם הוא רק יצליח להבין מה זה דגו אולי הוא ירצה צ’ינצ’ילה. בדיקה קצרה בגוגל העלתה שדגו היא צ’ינצ’ילה מדברית שיכולה לחלות בסכרת. רון חשב שהענין מרתק. נוספו לדגו יתרונות.

“למה לקרוא לחיה דגו אם היא פשוט צ’ינצ’ילה? זה בטח סתם rebranding“. איה בהתה בי ברחמים מהולים בהתנשאות קלה שביולוגים מפגינים כלפי שאר העולם. בנקודה הזאת החתול הג’ינג’י החליט לבוא למיטה ולהתחיל לעשות קולות של מטוס בואינג ממריא. משום מה החתול הזכיר לאישה פילים, והיא הגיעה למסקנה שהבית לא מספיק גדול בשביל פילים אז נצטרך לקנות שמורה. אני כמובן לא מרוויח מספיק בשביל לקנות שמורה, אז פשוט אצטרך לעבוד טיפה יותר, אולי לעשות איזה אקזיט או שניים. כמה מהר היא שכחה שאתמול היא הפצירה בי לעבוד פחות כדי שנוכל לבלות יותר זמן איכות ביחד. אם הייתי תופס את הבחור שהמציא את הביטוי “זמן איכות” בסמטה חשוכה הייתי מטפל בו טוב טוב.

משום מה היא התחילה לדבר על זמנים. “עבר, אתה יותר מדי קשור לעבר. You need to let go of the past. הרי למה שאמרתי אתמול אין שום קשר למה שאני אומרת היום.” לפחות היא מודעת לזה, חשבתי בטירוף. לעזאזל, עשיתי תזה בלוגיקה, אבל עדיין אני לא מתקרב בכלל להבין את מערכת האקסיומות וכללי ההיסק שהאישה פועלת לפיהם.

“רון, נמרים! אם הילדים יגדלו עם נמרים זאת תהיה חוויה מאוד חינוכית עבורם. הם ילמדו לתקשר עם נמרים מגיל צעיר, והם יוכלו לעשות להם בייביסיטר.” בשלב זה הזכרתי לאישה שאין לנו ילדים. “נו באמת רון, אתה קשור יותר מדי להווה. תתכנן קצת קדימה, תגדיל ראש!”
“טוב איה, אני מסכים להיפופוטם. את יכולה להזמין מחנות החיות.” לחצנו את היד על כך שאם היא מביאה היפופוטם, אין לי זכות להתלונן על כך. עכשיו רק שהיא לא תמצא מישהו מטורף מספיק בשביל באמת להביא לנו אחד.

כנראה שבאמת נצטרך לעבור דירה.

 

Alara Reborn Draft Walkthrough – 5-Color Naya

Another Alara Reborn draft, this time a Naya deck with some 5-color & bombs.

A lucky Alara Reborn walkthrough (Esper)

Check out this lucky Esper walkthrough (Alara Reborn).

Why I believe that 1≠2 more than I do the existence of God

Well, personally I am agnostic/atheist, but I’ve heard from several people that believe in God, that they believe “he exists” more than they believe that 1≠2.

Please put aside for one moment the exact meaning of the sentence “God exists”. All knowledge is, to an extent, subjective. I know that 1≠2 because it is a mathematical theorem, but also is a theorem that Zorn’s Lemma is provable from the Axiom of Choice – my level of confidence in these two theorems being true is rather different.

I claim that anyone (if he understands logic), even if he believes that “God exists”, should place more confidence in the statement “1≠2” than in his belief in God *. Why? Let’s assume 1=2, and prove God does not exists:

  1. Assume 1=2, and suppose by negation that God exists
  2. 1≠2 (from standard arithmetic axioms and inference rules)
  3. If God exists, then 1=2 and 1≠2 (from 1 and 2)
  4. Therefore, God does not exist (from 3, by Proof by Contradiction)

Q.E.D

So, one can prove from 1=2 that God exists, but I doubt you’ll find a proof that 1=2 from the assumption God exists. So believing 1=2 requires more faith, and provides more information, than believing God exists (because it implies God exists, but the other way around does not hold).

* Of course, there is at least one flaw in the above argument. Can you find it?

Thrift win32 binary

I’ve been playing around with Facebook Apache Thrift recently. I had a tough time finding working Thrift binaries for win32, and the compilation process was not trivial, so I’ve decided to put them online.

So here they are, compiled with cygwin from Thrift 0.1.0. This will probably require cygwin to run (remember to add cygwin binary directory to your path).

Note, the compilation process created two different files named thrift.exe – one small (16kb) file and the larger 10mb file I’ve put online (this one actually works).

Alt.net 2nd conference

I just attended my first alt.net conference (some would call it unconference). The story is about a group of 40 people that came to talk about … whatever they decided to talk about. The conference is self-organizing, with no predetermined lectures or lecturers, and with one healthy rule – if you don’t feel you are learning or contributing at the discussion you are currently having, you have to get up and find another discussion.

Here are some of the talks I attended (here is a semi-readable list of all the talks):

Aspect Oriented Programming

Usages other than logging, AOP frameworks.

Links: Cthru, Post#, Wicca.

Mocking/Stubbing

Reiterate the basic paradigm, emphasize on TypeMock. They are considering a UI tool adding to Visual Studio to help create mocks – meant for people just starting with mocking. The intended usage is:

  1. Write a test, without any mocking
  2. The test will usually fail because some deep class is not configured correctly.
  3. You will see the chain of calls that caused the exception, and be able to automatically generate a mock for any class in the chain.
  4. Rinse & repeat until your test passes

High Scale & Distributed Caches

The discussion focused around what I’d call medium scale – 2-10 nodes that used shared caches like memcached & Azure.

Multithreading

There was a comparison of Microsoft CCR and Parallel Extensions. It seems people still think of parallelization as simply utilizing all your cores, when it’s much more than that. Some applications benefit from multithreading even on a single core machine (think web crawler).

One interesting link – PowerThreading library (see this video for a demonstration of Asynchronous Programming Model using PowerThreading).

A survey of open APIs

I did some research in the past week on a few “Open APIs”, and wanted to share my findings here. This is just a summary of other, more comprehensive sources. Also, if you have any comments or corrections I’d love to hear them. I chose to present my findings as a list of concepts:

The Open Stack

This is an emerging stack of open protocols that will help build socially-connected websites. I will explore the key elements (I take XRDS-Simple to be rather low level and uninteresting).

OpenID

  • A single sign-on protocol (help to user not to create yet another set of user/pass)
  • Essential Workflow (here in more details):
    1. You want to logon to StackOverflow, which is an OpenID Consumer
    2. Instead of opening yet another account you are given an alternative (almost no site relies solely on OpenID).
    3. You either enter a URL (way less user friendly) or select from a fixed subset of Providers
    4. You are redirected to that URL, enter your credentials there (only if you are not logged in), and are asked to allow StackOverflow access to your OpenID identity.
    5. Depending on your OpenID provider, you can set for how long this access is granted
    6. Then, you are redirected back to StackOverflow, with a token (encoded in the URL), that is used to grant you access.
  • OpenID is mostly still just a login method today (doesn’t convey extra information beyond a logon grant) – although I did see some evidence to the contrary when I just opened an OpenID account at VeriSign – it seems websites can request more information from an OpenID provider – such as email, nickname and full name.
  • Microsoft, Yahoo, Google are now OpenID providers (in addition to more veteran providers). This is significant because it doesn’t force users to go to yet another place to open an OpenID account – they can just use an existing webmail account.
  • Facebook just joined OpenID foundation board (eBay is there already). Looks like it will become an OpenID provider, maybe also client.
  • Users are still not comfortable with it / unable to figure it out.
  • However, here is an interesting post about using email addresses as OpenID. When this happens, it might help bring in the users.
  • Here is a recipe to enable OpenID on a website (Consumer). Sample Provider implementation for .NET is here.
  • Of course, for your Wordpress blog the process is much easier – just install OpenID plugin.
    • Although, I heeded Tomer’s advice and instead used OpenID delegation – this means I use my blog’s URL as my OpenID, but it is just a redirect to a more serious OpenID provider. OpenID is/will be the keys to your world – better guard them safely.
  • Right now, the value of OpenID to a new website is still limited:
    1. Will not eliminate the need for us to implement user logon
    2. Not much value in being an OpenID Provider – it’s a nice to have feature, but in many cases not worth the cost (at least until you get a large user base).
    3. Is not sufficient, on its own, to get access to complete profile information about users (and use this data to help users interact with your website). But … can be complemented with more technologies.
  • Has a nice usage graph.

Google OpenSocial

  • A set of social APIs developed by Google (get list of friends, publish stories).
  • Implemneted y hi5, LinkedIn, MySpace, orkut, Yahoo!, Friendster (full list at the OpenSocial wiki)… but not Facebook (yet). All of these are OpenSocial Containers – data warehouses that answer OpenSocial queries.
  • OpenSocial is Open. The data is not stored on Google – the providers just conform to the OpenSocial API which reflects the data stored at each social network.
  • Had theoretical reach (not usage!) of 700M users Nov 08.
  • To serve OpenSocial:
  • Client implementations are available in javascript, java, .net, php, whatnot…

Google FriendConnect

  • An application that uses OpenSocial to enhance websites with social widgets (Comments, Reviews, …).
  • Is still not wildly spread (this directory is rather sparse at the moment)
  • Doesn’t seem to be targeted at big sites, rather at small sites/blogs (my impression at least).
  • No programming required to add social features to your site – but you have limited control over the effect.
  • Cool flow I tested: Login to FriendConnect on a website, and you already see in your friends list a buddy from Gmail that’s using FriendConnect at this site.

OAuth

Portable Contacts

Facebook Connect

  • Competitor of the open stack (OpenID+OpenSocial) – gives single sign-on + connects you to Facebook friends & feed
  • Uses a popup instead of redirecting to FB (less intimidating for users).
  • Has already been witnessed to boost new user signup.
  • Main Flow:
    1. User clicks Connect 
    2. Popup (in facebook.com) asks user to confirm
    3. User is shown a Presence Indicator at the target site
    4. Website can pull user’s profile & connections.
    5. Publish stories to Facebook.
    6. Send Facebook hashes of emails, and Facebook replies if they have a user with an identical hash. This can be used to show a user a count of his Facebook buddies that are using the target site (“10 of your friends are using this site, connect to Facebook now”).
  • Example of a FB connect-enabled site – TheRunAround.

RPX / JanRain

  • JanRain - An early OpenID player (in the market since 2005, one of the founders of OpenID).
  • RPXNow – abstracts away Single Sign-on, supports both Facebook Connect and major OpenID players.
  • Here is a blog post about why not to use it (Vendoer lock-in, single point of failure, too little benefit).
    • However, check out the counter arguments in the comments.
  • RPX get social profile data from Google, MySpace, Facebook.
    • This includes interesting profile fields like email/gender/birthday/photo.
  • The API also hints at getting an array of friends from relevant services.

That’s it for now, I hope you enjoyed this review. Remember that most of these APIs are very new or just being adopted, so expect changes to most of these. I expect a large API convergence to happen in the following year or two, which will simplify life for those of us building social applications.

Keep fighting the voodoo

Good developers dislike voodoo, but sometimes all the good will can’t stop a dark spawn of evil. For example, one of our build machines always bluescreens when running a certain system test. Why? Probably some hardware/driver failure, we’ll look into that.

In the meantime, let’s just ensure the build agent doesn’t get these nasty builds, shall we?

  1. Build Configuration –> Agent Requirementse –> Add requirements for environment variable –> SHITTY_AGENT (condition = “does not exist”)
  2. Go to the shitty agent and add a system variable named SHITTY_AGENT
  3. Restart the build service:
    1. net stop “TeamCity Build Agent Service”
    2. net start “TeamCity Build Agent Service”