Make it better, every day of the week

Read this

As programmers, if we haven't made our code base just a little bit better (preferably a whole lot better) for every new feature we add, we have failed. If no one but us can understand what a piece of code is doing or modify it in meaningful ways without bringing down the house, we have failed. If we take a shortcut/hack under pressure of deadlines or whatever but don't make it easy to spot/remove, we have failed.

As engineering management, if we don't deeply understand what makes our components good or bad, we have failed. If we can't set up processes and tools to monitor the health of our code, we have failed. If we can't motivate our teams to love their code and to want to make it better, we have failed.

More words don't make a better song. More brush strokes don't make a better painting. More features/LoC don't make a better product.

In his own (perhaps small) way, every good software engineer is a visionary. He pre-empts the future by building it today. When he can't predict the future, he build room in the present to handle it tomorrow. Our legacy is not the code we write today, but the code other will write using it tomorrow.

So what did you improve today?

The Official Solomon Says blog

Here's announcing  the official blog for Solomon Says.

I am no hotshot blogger, but I understand when Robert Scoble complains about noise. We follow blogs for some particular topics, and if the author starts spamming us with a whole bunch  of  other stuff, it becomes outright annoying.Over the last few months, I have devoted all of my time and energy to building Solomon Says into a useful website, and consequently my blogging has also taken on the form of a journal of my efforts. This was not a conscious decision, but I write what I am thinking about, and what I was thinking about was the site.

On the other hand, I felt compelled to share with my readers what I was planning with Solomon Says and the reasons for doing what I was doing. Since this blog was the only web destination I had at that time, this is where all the posts came. But some of you may be reading for entirely different reason and do not care about what changes I am making to the homepage. Other may only be interested in the progress of Solomon Says.

To avoid alienating both set of reader, I decided to separate the two data streams. Starting today, I am moving all blogging related to Solomon Says over to its own home, which will be the hub for all news, discussions, and brainstorming.

Go check it out, and subscribe if you would to stay abreast of what's happening in the world of online reviews.

Alway keep a backup. Of everything.

Let me tell you a story of failures and backups and pain.

So last night I finished a bunch of changes to Solomon Says.  After the regular load of testing (that lasts 15 minutes and includes opening a bunch of pages on Firefox and Chrome), I uploaded the changes and tried to bring the server back up. Everything exploded in my face at about the same time. The only reason we are still in business is that I had backups. In decreasing order of importance, the following backups saved the day:

1. Database
2. Code/Configurations
3. Images

So pretty much everything :)

At this point, a note on the deployment process is in order. Here’s how it goes:

1. Stop python fcgi process and nginx service.
2. Delete the production code.
3. Run the DB migration script.
4. Upload the entire code from my laptop to the production location.
5. Start python and nginx

#2, #3, and #4 didn’t go too well.

#2 – My dev. environment is Windows, but production is on LINUX.  So there’s a bunch of stuff related to path handling (‘/’ vs ‘\’ etc.) that I change just for development. This is automatically handled in production by using a different configuration file. Alas, I ran the delete for #2 from one level higher in the directory structure. Boom goes the config. And on bringing the server up, I get a load of ‘access permission denied’ errors. I spent a half hour analyzing the arcane debug messages, then give up and restore the entire code base file by file and change by change.

#3 – I missed selecting a couple of ‘where’ conditions when running the migration script. Result – 2 of the main table got randomly changed. Considering how crappy the day had been so far, I realized it only on restarting the server. So bring the server down again, restore the DB to its previous avatar from the backup, and run the migration with extra precaution.

#4 – My development copy did not have quite a few of the images related to the newer reviews I had posted. And since I had deleted the production data in #2, the server started throwing ‘Suspicious Operation’ exception (What the hell is that? It should have said ‘File not found” or something). In view of the blunders I had made for #2 and #3, I assumed a mistake in the new configuration I had created and spent another hour debugging, then gave up and copied over the image folders from the back up to production.
 

All told, something that should have taken 15 minutes took 4 hours.
 

Lesson learnt. Always keep a backup. Of everything.

Looking just a little bit better

A lot of the feedback that I have received on Solomon Says (a big thank you to everyone who spent time and effort providing it) has been regarding the styling and design aspect of the website. Or rather, the lack of it styling and design aspects in the website. Now, I am no designer. CSS3 and templating were not quite my fortes when I started working on it. So operating out of my ignorance of these fields, I have been forced to improve the design of the site in increments. Get something working, make it usable, and put it out there. Then improve what it looks like in the next iteration.

I'd like to share with you some of the changes I'm currently working on to the layout of the review pages. This primarily involves improving the data panel just above the text of the review. For the uninitiated, this is what it currently looks like on book and travel reviews respectively.

Both look very cramped and difficult to interact with. The huge orange rating section is sort of a waste of space, and the images don’t get due prominence (especially harmful on travel reviews). So I thought through these problems and came up with a small redesign which hopefully makes everything cleaner and easier to access. Check it out.

The new version is only slightly different from the current one but I think it lends a much more spaced-out feel to the whole page. You would also have noted that there is a small panel of image thumbnails right above the ratings section. These are the images that currently show up below the text of the review, like so:

I never really this design because pics are cool and everybody loves them. So I moved the images right to the top in a combination of sliding thumbnail carousel and Fancybox. Now they are easily accessible, and clicking on the thumbnail gallery blows them up to full size too! Like this.

A lot more groovy, even if I say so myself! I am planning to roll out the changes in about two weeks after a few minor tweaks and testing.

So what do you think? Like the new look? Not quite? Let me know in the comments or drop me a line at solomonsaysindia@gmail.com. Suggestions/flowery words of praise/hate mail are all welcome.

Solomon Says at ISB

First things first – Please fill out this short survey. This will help me in assessing what I can do to make Solomon Says more exciting and useful for its users. I really, really appreciate it.

Now for the news of the week.

Solomon Says is currently the subject of a marketing project/case study in a course on Entrepreneurial Decision Making (EnDM) at ISB (Indian School of Business). The project is being conducted by Varun Jain (a very close friend of mine from my undergrad days at NSIT) of the ISB class of 2013 under Prof. Arun Pereira. Over the course of the project, I will be working with the two aforementioned gentlemen (mostly with Varun) to conduct market surveys, audiences analysis and other analytical wizardry to refine SolomonSays into an even more awesome product.

Quick background on how this came about. Essentially Varun was looking for a start-up to whet his new-found marketing chops on.. I was going around writing reviews and hacking away to glory with no time for reaching out to the wide world and finding a place in it. We discussed the website one day, and agreed that it could use some MBA lovin’. So starting this week with survey mentioned above, we’ll be doing some basic scoping exercises to (hopefully) understand out audience and define our market with a lot more clarity than before. These efforts will also try to discover how readers interact with the website and what we can build into it to make that experience smooth.

I have written before that I do not have a proper business plan yet for SolomonSays. Throughout this ISB affair, my focus will continue to be on how to make this the best, most helpful reviews website on the web. No doubt there are parts of the project which demand an emphasis on revenue streams and sustainability, but those come later. Till then, the spotlight, my dear readers, is on you.

Don't forget the survey.

Meet SolomonSays : All the review you will ever need

This post has been coming for a few months, but somehow I could never get around to writing it. But a certain milestone has been reached today that makes me want to sit down and write this. A minor milestone, no doubt, but something is better than nothing right? Over the last 10 months or so I have been working on a website called SolomonSays. This is a blog-in-website-shape where I review books, travel destinations, and eateries (though I am yet to get any traction on the last one). The milestone – 100 likes on the facebook page.

I have referred earlier to the broken-ness of the crowd-sourced or community driven model of getting reviews of products. It is great at getting lots of info and scales insanely, but none of this data is in a very usable form (short of going through tons of reviews). And apparently, I’m not the only one cribbing either. The other problem is that one has to go to so many different sites to get reviews on different things despite the fact that all these websites are doing the exact same thing – gathering user generated review content.

I first faced this problem while looking up book recommendations and reviews. So I set out to do what I thought was the right of reviewing things. Which, I believe, is the old fashioned way of having a team of dedicated reviewers writing comprehensive reviews about whatever it is that is being reviewed. I believe that no one should have to go through 300 one-off personal experiences to get an idea what a new restaurant or car is like. These are critical elements of a discussion, but do not comprise a coherent review. Combining the two approaches, i.e. getting a community to offer their individual opinions around a central review, is the clear winner IMO.

My original goal was extraordinarily ambitious – SolomonSays.in would review absolutely _everything_ that can be reviewed. Books, Restaurants, Home appliances, vehicles, were all fair game. While I realized that actually doing this wouldn’t be possible all at once, that was where I believed I wanted to get to. Such a website, if it could be created, would occupy a unique position between Wikipedia and Amazon – the reviews would be free, and we would be able to facilitate any purchase the reader wanted through affiliate partners. But the content would be the business. Data would come first. ALWAYS. I still hold to the original unbounded scope of the website - but getting there is being done only in baby steps.

All MBA and otherwise savvy reader will have seen that the above isn’t much of a business plan – write reviews till you have enough content to drive steady traffic through SEO, and then try to convert that traffic into sales. In all honesty, I didn’t think too much in terms of a business idea (I still don’t). I was extremely excited by the idea of having all this data under one roof, the day job wasn’t too interesting at the time, so I just ran with it. One of my friends whom I initially discussed this with still hasn’t stopped harassing me about monetization.

The time spent working on this has been very exciting, and very tiring. I essentially have two jobs, so I get to hang out with my friends a lot less that I used to. For all that, the learning has been tremendous (I have written earlier about the technical experience). But now I am paid to read books (not yet really, but soon will be, I hope J), and writing about what I read clears things up in my head too. Besides, there’s no better feeling than when drops a line saying how useful they found something that I wrote. 

At this small occasion, I thank everyone who has supported me thus far (Special mention – Aditya Mangla, Sid Reddy, Sreejith, Kalpana, and Sin City). Do drop me a line with your ideas, suggestions, and review requests (or if you are willing to share your beer). Random gossip and philosophical musings are welcome too. 

Whatever I code or write is only half the answer. The rest of the awesomeness comes from your experiences and participation. I will continue to bring more information to this party. Hope you’ll stay. Let’s clean up this morass of online reviewing!

The problem with crowdsourced content

We have whole lot of sites featuring reviews written by their readers on pretty much all topics. Of these, at least the front runners attract large traffic volumes, have a devoted user base, and are, I assume, making money. However, I want to discuss what I think is wrong with them.

The problem is the method of generating content. All these sites rely on a bevy of users to come write reviews on the products/service that the site focuses on and that they have used/experienced. This can be restaurant, book, travel, gadgets, or whatever else. Via friendly UIs, facilitated social media interaction etc., users are encouraged to contribute data for each others benefit. This strategy is very effective in generating large amounts of data. However, it is very bad at generating cohesive data.

In general, I have three somewhat interconnected problems with this:

1. Data is of poor quality – Since the website wants readers to submit reviews, it can rarely hold them accountable to the quality of their writing. The aim is to lower the barrier to writing and social media sharing. Get him to write. No matter he writes, get him to write. To be fair, most of the reputed website will intervene if you write inflammatory or profane material, but apart from that, pretty much anything goes. As a direct consequence, the quality of review in terms of both the content (what is written) and the form (how it is written) goes down. Most people write unbalanced reviews, either giving full marks and endless praise or griping about a very bad experience.This can be avoided to some extent by making sincere efforts at moderating and community building (StackOverflow is a great example), but none of the major commercial websites seem to be doing so.
2. There is just too much data – This is the explicit result of successfully inducing readers to submit content to a site. Since anyone can submit anything, the data volumes are large, and it becomes well-nigh impossible to find information. This is what I like to call the Problem of 500 reviews. What I mean by that is that on any successful reviews site, today you can find 500 reviews for pretty much every single item. Too much data is not much better than no data. The best this deluge of disjointed reader inputs can give us is a general sense of how people like something. As an experiment, choose any famous book about which you know nothing, and try to find out about the book using only GoodReads reviews. I am fairly regular on the site, but I mostly do it for the bragging rights, and to let my friends know what I am up to.
3. Data is without structure – I totally agree that a successful travel site of the kind that we are talking about will have all data about some destination. But how do we find this data? Since it is broken up across a large number of unconnected reviews, it is very difficult to present the information in a coherent, intuitive manner. It is now left to the reader to sift through the data that each reviewer has provided and collate the data he needs (when to get there, how to get there etc.)

The crowdsourced content model is like a group discussion where everyone is talking at the same time. There is no anchor or reference around which a discussion can be built.

IMO, a far better alternative is to have an informed member write one review, and then use that to gather all sorts of varied and personalized experiences regarding the topic of discussion. It may seem so up front, but such a model (critic-driven model, if you will) is not about classroom style information broadcast. The Expert has not spoken. It is about providing a structured core, the basic information, and then inviting the readers to extend that into a wider body of information. If you want people to spend time and effort sharing their opinion, it is only fair that you offer them something in return.

A sample Ladakh itinerary and a vote of thanks

So yours truly went to Leh two weeks ago. It was, as expected, a mind-blastingly awesome experience.

It is not the most routine of trips, so I pulled up my Google socks and got down to business. Although data is widely available on each individual destination in Leh, I found difficult to plan an itinerary since no one was very clear how long each place would take (I found Nubra Valley estimates from "Dont even bother going there" to "It will definitely take 4 days"). The tour operators will certainly make one for you, but the geeks (and the jock for that matter) _has_ to work it out for himself, right?

I eventually had to call up friends who had been there to get their itineraries. Not cool. I don't want to be real life social when I can be online social. So to save all you worthies this trouble, here is my itinerary:

Day 1

Arrive Leh and overnight Stay. Check out Leh Palace and Shanti Stupa. Chill in random cafes.

 

Day 2
Leh-Like Monastery - Gurudwaara Pathhar Sahib - Lamayuru Monastery-Ule :  

You can actually visit Alchi the same day and come back if you rush somewhat, but why bother? The nice Ule resorts are built overlooking the river. Ergo, overnight stay.

 

Day 3

Ule-Alchi-Leh : Alchi monastery is nice and old. You can visit Basgo Palace on the way back, but it is really just a pile of mud and stone now and not worth the climb. Also see Hall of Fame - an army memorial/museum for our high altitude warriors. Respect.

  

Day 4

Leh to Nubra Valley - The drive is awesome. Bragging rights are earned by clicking pics at Khardungla (the highest motorable pass in the world at 5600-ish metres). Then bactrian camel ride at Diskit. It's  beautiful here and a hard ride back, so stay overnight.

Day 5

Drive back to Leh. Relax.

 

Day 6

Leh - Chang La - Pangong Lake : Changla is the third highest motorable pass in the world (14500 ft. Reach camp around lunchtime. Sit on the banks of the beautiful lake. It's frikking cold and windy.

 

Day 7

Pangong Lake - Leh. Enroute visit Hemis, Thikse Monasteries and Shey Palace.

 

Day 8

Go home

 

Everything was done at a mellow pace, no rushing. Spending less time than this would not be doing justice to them IMO. Also, this is a fairly touristy plan, since this was my first time.

For all the research, we still booked a packaged tour (I'll hulk out the next time, I promise) via Escapades India. That almost got effed up due to the recent flash floods. We were stuck in Manali for 2 days, and missed the first two days of our itinerary. But the folks over at Escapades were EXTREMELY helpful, stayed in touch constantly to find out if we were doing okay. I called the unputdownable Mr. Parag more times than I called my family and he just might have done the same! A mail of theirs said "On adventure trips, you want to worry more about the reliability of your operator rather than what he is charging you" (paraphrased). Parag and Co. certainly came through on that front.

We extended our trip by two days to compensate for the things we had missed, and Escapades arranged it for no extra charge. Read that again. Slowly. No extra charge.

So all in all I strongly recommend anyone looking for the arranged travel option to go to Leh (or elsewhere, but I'm not very sure of  that) to get in touch with Escapades India.

Chance knowledge and the cumulative effect

As I was driving past a park this Saturday, I saw that a section of it was burning. I suddenly had a vivid image of the entire park burning with high flames (It is a very large park). This then caused a thought about how Rome would have looked to Nero as it burnt. The revelation – I now know why Nero (the software) is so named.

This sort of random revelation is not an unusual thing. The most important part here was that I wasn’t thinking about computers at all. The experience was quite Sherlock Holmes-ish – When he gives voice to Dr. Watson’s thoughts by saying “Such a waste” and then proceeds to explain the chain of thought in the latter’s mind. Richard Bach compared ideas to fractures running in a crystal – any could lead to any other. Others have referred to the oneness of knowledge – that given one thing, everything else could be discovered. Both seem somewhat limited ways of explaining the experience.

Chance discoveries happen in science all the time. Microwave background radiation and radioactivity  are but two examples.

I’m thinking now of the cumulative nature of scientific knowledge. Both socio-economic theorists (Farcis Fukuyama, Fareed Zakaria etc.) and scientists (Francis Bacon) assert that with the invention of the scientific method (hypothesis-experiment-prove/disprove), scientific knowledge has become cumulative. That later generation inherit the knowledge of their predecessors. And it is intuitively true. We don’t have to rediscover the gravitational principle, although we may have to prove it several times over (damned CBSE exams!).

This is brought into somewhat contrary focus by the revelational experience. The cumulative nature of the scientific method no doubt holds true when the steps are small. Not to belittle any discoveries, but some _are_ greater than others. It took a Newton to get us to gravity, and even more drastically, an Einstein to get to relativity. Both works are quite out of the league for their times. The tools they used were there, as was the a-priori knowledge. But the power of the method seems to wane when we consider efforts where accepted first principles have to be discarded.

The cumulativity assertion says that given a state of human knowledge and the scientific method – the future can be worked out again. But how does this apply when the prior knowledge has to be discarded by a leap of faith.  When accidents force the next step forward, how does the scientific principle handle it? We can say that the geniuses use informed intuition (Kekule and the structure of Benzene). But that would lead us into the myriad definitions of genius and intuition.

And that’s a discussion for some other time.

Technology, not activism, will save the environment

A lot of people are interested in preserving the environment. I admit I don’t do much about it (I’m more the armchair sort), but I am interested in it. To those who are out there doing it (planting trees, spotfixing etc.) - more power to you!! You have my respect. This post has nothing to do with you. This is about the people who are corporate bitches of the “sustainable business practices” variety. I will say this out loud and clearly – That shit don’t work. Let me explain why.

What does “sustainable business practices” mean? It means that we will continue to do business with just about enough tweaks to it so that it allows the self-same business to run a little longer. If the tagline gets us a few tax breaks, so much the better. I don’t know of any company which has drastically changed business models or made business decisions on the basis of sustainability. Sustainability is good PR, but it is faaaaaar down the table from the bottom line. The jobs I referred to exist for posterity, and to employ mavericks who are willing to work for low salaries and pointless objectives.

Think about it. In the first place, the concept that we can somehow undo the damage we have done to the environment is kind of suspect. We can stop it from getting worse if we abandon all polluting activities right this instant, but the ecology of a planet is an immense beast, and is not easily turned from its path. Apart from that, the scale at which we are screwing things up is huge. Optimizations to the existing products/processes can only buy some time before the jig is up. Besides, let’s face it – corporates don’t truly care about environment. This is not to say that people working there don’t care, but usually when “The firm comes first”, clean air comes last.

Henry Ford (the first) said – “For an idea to work, it has to be right in time, and it has to be right in price”. Meaning someone making very expensive horse-buggies in 1900 would be wrong in price, and someone making very cheap horse-buggies today would be wrong in time.  Neither would work. Today, the idea of environmental conservation is right in time, but I think that it is still wrong in price. We simply do not have economically viable substitutes for most of the stuff that is going wrong. This is why I firmly believe that the tide will turn not by activism, but by technology.

Activism is a fine thing, but it is not the solution here. It simply does not scale well. What we need is new, clean technology that performs at least as well as the current polluting setup. Only once we have these alternates is when the battle can be joined in the earnest by switching businesses to use them and making the enterprises actually sustainable.

New products that are cheap and yet eco-friendly will be immensely disruptive to the existing system. This is historically consistent (significant changes in living conditions occurs with significant change in technology) and the only hope that we have of turning the development juggernaut on its head and setting it in the right (or at least somewhat) direction. But since it is disruptive, it will face opposition, and THEN we will need environmental managers to carry the standard into boardrooms. But first that solution(s) that can be peddled has to be discovered.

So, here’s my tuppence – If you truly, deeply care about environment, go study the sciences. Try to actually solve the problem. Don’t waste your time and effort getting a corporate degree in Environmental Practices (or whatever they call it). It will be irrelevant and useless.