Hackterms, Pt II

I had spent some time trying to get Hackterms off the ground, and then stalled and left the project sit for a month. I had spent days agonizing over small design decisions – lining up icons and such –  and then had a single, successful push for my first 20 users.

And then I stalled. Interestingly, this slowdown didn’t come from negative feedback or a lack of interest. I expected harsh user feedback, trolls, needing to cut/revamp features, user churn – in a word, users not caring about the site and not giving it the time of day. I was prepared for it. However, none of that really happened. A few of my users were moderately interested, and then I ran into a few pretty complex bugs – for example, the Submit Definition button wouldn’t work on iOS. Early adopters were still responding on my Slack channel. Though I always responded, I often didn’t come through. I kept promising to return to fix the bugs, and then kept stalling and kicking the ball down the road.

However, unlike some of my past projects, I never considered abandoning Hackterms. Even after about a month of inactivity, I thought about the site frequently, and – between encouragement from friends and belief that this project should be out in the world – was itching for a chance to come back to it. Stepping away from working on the problem daily gave me space to think deeply and reconsider my approach.

When I started, I attacked Hackterms enthusiastically, jumping into the fray (as I tend to) and building, designing, and planning all in one. Although I  wireframed and set up a kanban-style Trello board, prioritized features, and otherwise tried to plan ahead, the truth was, my structure was convoluted, and I had made a lot of imperfect technical decisions. For example, I was using a mix of EJS and Handlebars.js to render definitions and comments. When a Slack user asked how he could try to contribute to the code, I attempted to explain the data flow to him and realized it was anything but intuitive. To be honest, I myself was a little worried about revamping parts of the site because I couldn’t remember how I built them myself.

Sloppiness like this is the tradeoff for building quickly, but I had gotten to a point where my core features were up, and it was time to do two things:

  1. get some serious users (I arbitrarily decided on 100 unique users and a 1000 definitions as my goal). I was at 20 users thus far.
  2. revamp my code to make it readable, accessible, and ready for collaboration. Esther suggested open-sourcing the project (which was also the top-comment Reddit request), and I wanted to be able to get James’s help with my code.  I was anxious about this, but I knew that it was important to clean the code up.

I was reading the excellent Lean Product Playbook (highly recommend!) and kept thinking about how I could apply my learning on Hackterms. The book explained a lot of the topics I sort-of-intuitively half-assed in a very structured and insightful way. No matter how much it sucked to admit, I was making the fatal mistake – I was coding a massive project (telling myself it was an MVP) before getting real user feedback. My sort-of proof of concept was a massive encouraging reddit thread, and a few initial users. I was kind of following a lot of Lean principles, but not in any structured way. The book encouraged me to come up with hypotheses about who’d use the site, brainstorm a set of features, select my top features, categorize them using the Kano model, and then test them.

I had done  this, kind-of, sort-of.

  • I had come up with a value prop based on my own experience: users want to quickly identify what a coding term does and where it fits in
  • I came up with a minimal feature set: users need to be able to search for and view definitions, and also add new definitions for a given term
  • I then added with what Kano calls a “delighter”: instant search (search/display with every letter you type, just like Google!)
  • After that, I went off the deep end, and decided to build a moderation system that queues up new posts for approval. This was not necessary for an MVP at all. I was anticipating a problem that might come with scaling – trolling and irrelevant content. However, I actually spent forever building this.
  • My proof of concept was effectively marketing: lots of Redditors said “this is a great idea!” I then got a few of them to join after I built the product.
  • I conducted pretty extensive qualitative research with my closest friends. I sat next to them and watched them used

In my last job at Codecademy, one of my managers repeatedly disparaged me from relying too much on intuition in decision-making. Reading the Lean Product Playbook, along with the time that I stepped away from Hackterms reinforced just how important planning, forming hypotheses, testing them, and using metrics is. Up till this point, I’ve been (quite frankly) a bit confused about the relationship between data and intuition in product. Data seemed like the “right way” to make decisions, but we also glorify brilliant product visionaries (you know exactly where this links leads) and the idea of solving your own problems (which may not be other people’s problems).

The book really set me straight on this, coalescing a lot of the advice I’ve heard (but not understood from others). Early on in the product, you rely on intuition. I used to picture this as closing your eyes and visualizing/brainstorming brilliant ideas – but, really, you form hypotheses based on your (often unconscious) understanding of the space. That’s why solving your own problems makes sense – you don’t have to guess at others’ pain points, because you know your own. Anyway, you come up with hypotheses, and then you build some version of the product ( your MVP), and then you test it – and rely on data to tell you if the product is working (in other words, whether you’ve achieved the mythical product-market fit).

I wasn’t totally wrong – I just attributed way too much mystery and magic to “getting it right with intuition”. I learned that no matter how you come up with your product hypothesis, you need to figure out (1) exactly what you’re testing (2) what core metric represents this – Dan Olsen says that early on, this is usually user retention, and (3) who your target users are. Basically, I needed to create a framework within which I could be creative; direction to my decision-making.

I had a strong personal problem that I wanted to solve with Hackterms, and a lot of intuition around why others might need my product (that I developed spending time around people learning to code), and roughly knew that more users and definitions is good. But, but I never properly formulated my hypothesis, picked a metric to test, and put it in front of users.


Around this time, I stumbled on Courtland Allen’s interview for a YC podcast, which in turn brought me to Indie Hackers, and then IH’s interview with Pieter Levelsand one of Pieter’s own talks. This seriously kicked my ass – especially his advice on validating by launching.

I knew because I spent one year on this Tubelytics app and that didn’t work so I was like, okay I shouldn’t have spent so much time. I should have done more startup stuff just ship, and deploy, and launch, and validate, this is my idea and this was definitely new or my perspective on it was new, validate by launching. You don’t know if the app is going to work but you need to launch it, see if it’s going to work and you need to understand that most apps won’t work. I knew that, so I was like, “Okay I’ll just do a lot and I’ll see what sticks.” like throwing spaghetti at the wall and then I started making one thing a month.

I had spent nearly four months building Hackterms on and off, and I knew strongly that I needed to validate or move on. So, I finally did something I’ve been meaning to do for months: I invited my friends James and Jon to add some more seed content. We sat around a table, ate pizza, and wrote content – and talked about what good content looks like. We ended the evening at ~125 definitions. I decided it was time to share Hackterms with the world, and deal with whatever the consequences are.

If users asked for changes, I’d make them. If the website flopped, I’d move on.

I put the finishing touches on the last muse feature – a list of most searched and most requested terms, and then shared the post on Indie Hackers. I waited eagerly all day, aaaaaaand…. nobody responded. People visited the site, looked up the top searched terms, and then left. No new definitions added, no comments.

The next morning, I got up early, and posted to Indie Hackers again, this time using the “Urban Dictionary for coding terms” tagline, and also posted on r/webdev (which was technically against their subreddit rules due to timing.)

giphyIndiehackers stayed dead silent, but Reddit gave me an enthusiastic welcome back. Despite the fact that I kept waiting for my post to be removed (because you could only show off projects on Saturdays, and it was a Tuesday), I suddenly had more users on my site than I ever had before.

Things got a little wild as I watched the user numbers climb up and more user dots appear on the global map. I was getting a ton of upvotes and a ton of really great comments. Amazingly, people were signing up and adding great definitions. The number one suggestion was “add oAuth”, which I quickly learned meant one-click Google login.

Screen Shot 2018-02-13 at 11.34.29 AM

I hit the top post on r/webdev, and the site briefly went down at about ~80 simultaneous users, which left me absolutely terrified – because, frankly, I had no idea what to do. Fortunately, after I restarted the server, it seemed to continue working on its own, and I thanked my lucky stars. Someone even posted the link to Hacker News, which I was mildly terrified to do myself – they’re serious technical people over there!

Screen Shot 2018-02-13 at 10.41.04 AM

And then… my run for the day was abruptly over. My post was removed, because it wasn’t Show Off Saturday or r/webdev – as I mentioned earlier, it was Tuesday. In a way, though, this was good – it gave me a good taste of what having lots of simultaneous users is like, and I got some great feature suggestions.


The #1 request I got was to implement oAuth in order to allow users to log in with Google/Github, so I got to working on this. Integrating these was a bit of a pain, so I wrote a whole another blog post on this – but managed to get it working! Additionally, I spent more time coding with Esther,  and further tweaked my design based on her feedback. Every time I looked back at my older designs, the site felt more and more out dated – and consequently more modern with every tweak. So, I wondered how outdated the site looked now, and how I could improve it. I kept paying attention to designs I’d see on the web, and integrating little tweaks to my site. James pinged-ponged my own advice back at me – don’t try to reinvent the wheel. How do other dictionary sites handle their design? It was good advice, and I followed it.

Around this time, I was about to start two different jobs – one was a long-term contract, and the other a weekend consulting gig. I knew I wouldn’t have the luxury of working on Hackterms as intensely as I have been, so it was time to get my user #s up. I had received validation from my previous launch, but it still felt like luck – like my users were doing me a favor. A few weeks later, motivated  by an impending deadline, I decided to launch again, I knew I’d have less and less time, so it was now or never. My girlfriend and I were visiting friends in Boston to celebrate a birthday that weekend, and I finished oAuth for Github and and Google that Saturday morning. I knew I’d have little time to participate in the ensuing discussions, but I was slated to spend the next 4-5 weekends working, I didn’t have much of a choice. I decided to launch and trust the site to hold its own.

I once again posted on reddit on both r/learnprogramming and r/webdev, and… once again, the post exploded. Except, this time, it exploded much more on r/learnprogramming than on r/webdev.

Screen Shot 2018-04-02 at 12.22.01 PM

I was sitting in my friend’s living room, and I had my laptop open, trying my best not to get distracted from the joyous hangout. After a while, they became curious about the climbing numbers, and I briefly told them about the site.  We were watching my post climb the rankings, and the user numbers creep towards 50, and then 80 – where the site leveled off before. From my previous crash, I assumed that the reddit hug of death would be me eventually, and I didn’t know why or how to fix it. I was getting more terms and signups, and the post was at maybe 300 upvotes; it was an amazing feeling, and it was great to know my last launch wasn’t a fluke.

As steam picked up, we watched the number climb closer and close to 100. And then, the site crashed. Without skipping a beat, I restarted the servers. We were back up. Except, this time, I glanced at the log. It looked like the crash was cause by a regex parsing error. The site stayed up, and I went back to playing our board game. Except, ten minutes later, the site crashed again. I checked the logs – same error. This was great news. The Hug Of Death wasn’t bringing my site down – a simple code issue was. I excused myself for a few minutes and dug into the code. The issue stemmed from a small search function that converted non-alphanumeric characters into ones digestible by URLs. The simplest thing was to just comment it out – this would impact search results for terms like C++ and A*, but in return, would prevent my site from crashing. Worth it.

I pushed the fix, and held my breath as I went back to playing the board game. No more crashes.

And then, to my disbelief and my friends’ delight,Screen Shot 2018-02-24 at 6.02.05 PM.pngwe crossed 100 simultaneous users. It was a completely arbitrary (vanity) benchmark, but it still felt surreal to see. Over 100 users were on Hackterms, and it wasn’t crashing anymore. The dreaded Hug of Death wasn’t getting me, and I managed to isolate and fix the coding errors that brought me down earlier. Positive comments were pouring in. Lots of users were signing up to create new definitions.

In a flattering move, I even got trolled – which I took as a huge compliment. Users caught wind of the fact that searching things would add them to the “most searched” panel, and … rickrolled me. I got a good laugh out of it; it hurt me to take these down, but I knew I needed to keep the integrity of the content.

Screen Shot 2018-02-25 at 1.53.22 AM

Never gonna run around… and desert you!

 

Two more noteworthy things happened that day.

First, because the population of r/learnprogramming is by design made up of beginners, I got a lot of visits and user signups, but not nearly as many definitions. My user count jumped from 54 to 250+. This stumped (and encouraged) me, because I couldn’t see a reason for people to sign up other than to add definitions. However, due to this huge ratio of learners, I got an influx of searches, but only about ~300 new definitions in the next few days (which was amazing, but also a much smaller ratio than my r/webdev launch).

Second, I got a crash course on internet security. Someone spammed the site with XSS, which I knew nothing about, and users were reporting crashes. Users had injected JS through my definitions, causing all sorts of alerts to run – including one that caused an infinite loop: while(true){ alert() }

I felt totally out of my depth, but with a bit of research, quickly sanitized input and removed the definitions. In the grand scheme of things, this was an important lesson, but (thankfully) didn’t majorly impact the launch. Later, I found that users had been complaining about popups/weird terms on my /all page throughout the day, but I didn’t see it till the evening.

A few users even emailed me to help me fix several security loopholes – including a clever one where a user managed to impersonate me with invisible characters. It was a really nice thing to do.

It was fascinating to watch the change in development priorities. While building the MVP, I tried to focus on validating crucial features, asking do people need this? Now that lots of users were on the site, a very different set of priorities was becoming relevant. Where is your privacy policy? Can I delete my past submissions? Is the input sanitized/secured? Why aren’t you running an HTTPS site? Do you have a docker image I can contribute to?

All in all, I ended the weekend with ~350 definitions and lots of new users. The launch was clearly a success, and contributors – if not learners – were eager to share their knowledge.


Over the next few days, residual definitions kept coming in. I launched on Saturday, and the following Sunday to Tuesday, users continued to frequent the site, catching up on reddit threads from the weekend. A few days later, I was out to dinner with James, and was telling him about the launch. I told him that I got a bunch of Twitter mentions I didn’t know about, and was showing those to him, when an article – posted just hours before – caught my eye. Dice, a job search site, had written about Hackterms. Reading through the review, I was floored. It was positive press, and it was spot on with what the site was trying to do:

… the crowdsourced definitions are far easier to grasp than Wikipedia or general web searches. Written by developers and engineers who probably have to explain these terms regularly seems to be a winning strategy.

I was amazed and humbled and floored. I wrote the author to thank him for the article (and  not-so-subtly asked if hew knew anybody else who might want to write about the site), and I continued personally emailing each new Hackterms contributor. A few days later, we got another writeup by JAXenter, a java-focused tech publication – again, spot on about the mission.

Obviously, programmers have personal preferences about what tools and frameworks they uses. However, Hackterms makes no judgement calls as to the superiority of one system or another. They also don’t go over code snippets or examples. This is a dictionary, not a how-to explainer. The goal here is to provide definitions as clear and concise as possible.

I was elated, and took a few days to think about what happens next. 1000 terms seemed like a good short-term goal to aim for. It was clear that devs want to teach – but would Hackterms be helpful to learners? I needed to figure this out.


James suggested a great thing, which was echoed by the Dice article. He said I should tap into individual coding communities – Swift, and Haskell, and Rails, for example. My job start date got pushed back a week, so I had another Saturday to launch – and I couldn’t let it go to waste. I looked up the top programming subreddits, and realized I never launched in the biggest one: r/programming. So, a few days later, I posted on a couple more subreddits – r/programming, as well as r/iOS, a WordPress, and a Haskell subreddit. This time, I linked the Dice article, to give the site some legitimacy, and (frankly), to show off a little.

The post started picking up steam in the morning. As new definitions came in, I would routinely go into Mlab (where I was storing my mongo database) and combine/merge certain terms in order to keep the definitions consistent (for example, merging backend and back-end, or node.js  and nodeJS). I was making one of these changes, half-absentmindedly (I was in a pretty bad mood that day), when suddenly I saw my realtime user numbers fall from 80 to 25. I saved a term I was working on in Mlab, which refreshed it, and showed me a snapshot of my database. I glanced at it, and then stared at it in horror as I slowly realized…

… my term count was 0.

I opened the site, and there was no content. Thank god, my definitions seemed safe, but given how I structured my data, definitions couldn’t be pulled up without terms. I had just hit 375 terms (from ~330 at launch an hour ago), and all my terms were all gone. I refreshed the site a few times as the reality set it.

For about two minutes, I genuinely thought Hackterms was done for. For the hundreds of hours I had put into coding the features, the true value of my website was its content. Hundreds of people took the time to write thoughtful definitions, and now there was no way to access them. Never mind that my launch was an embarrassment – there was no longer any content on the site; I had let down all the users who took time to make accounts and contribute. All that work, done for.

(I later found out that I most likely caused my own error. Absentmindedly, instead of deleting one duplicate term record, I deleted all of them.)

crash

I consciously had to grab myself by the back of my shirt (figuratively, of course), and pull myself out of the downward spiral I had entered. I wasn’t going down without a fight. I was tempted to look at the reddit thread – no doubt, dozens of messages about how my site is broken were coming in, but instead, I stopped and thought.

I had no backup. But, thank god, the definitions were safe. The core of my content was fine. And, each definition had a term field. So, if I could somehow pull the terms from the definitions and create 375 new records from them, my site would be up.

I wasn’t going to go down without a fight. I knew there was no instant fix, and the launch was botched – but not over. It was only 10AM; the post would be up for a while. As tempted as I was to look at the repercussions of my crash on reddit and feel bad for myself (it was bad – I was having a bad day, and I just wanted to sulk,) I knew I needed to focus 100% of my attention on fixing this mess – and there was a glimmer of hope. I brought the site down for maintenance on Heroku.

My first thought was to try and rebuild the definitions manually, by brute force, but 375 records would take me forever to create. Plus, each mongo record has a unique ID, and I couldn’t make that up – I’d need to add each record one by one in Terminal. No, that wouldn’t work. A better way would be to cycle through each definition, and create a term based on the term field. I could test for duplicates (since some terms had multiple definitions) as I created the records.

The code would be messy, but not difficult. I knew how to do it. I sat down and began writing code, doing my best to quiet any negative thoughts swirling in my head. At the surface, the code wasn’t difficult:

  1. cycle through each definitions
  2. grab the term attribute and search the database for it
  3. if the term didn’t exists in the DB (search.length == 0), create term
  4. move on to next definition

I went into my local environment, which had ~75 fake definitions, and deleted my terms collection to recreate the error. I wrote the code quickly and with surprisingly few errors. I kept trying the simplest solutions I could think of – to cycle through my definitions, I used a for loop to read each term, and if there was no such term, create it. Mongo threw an error – the for loop was executing almost instantly, so I was creating too many records in a very short time. Simplest solution? SetTimeout. After getting a quick solution to the classic for loop/setTimeout quagmire, I ran the code locally. Amazingly, it worked. I knew I could afford to fuck up a bit – if I accidentally created duplicate terms, or even most terms, I’d be okay. I could clean it up, fill in the holes. I just needed to automate the bulk of the work. I named the trigger button “restore database”, ensured it was only accessible to me, and gave it the id of “Jesus” (to bring my data back from the dead.)

I pushed my code to production, and checked Heroku logs, then turned Heroku maintenance off; the site was live once again, with no content. Very quickly, I started getting a ton of traffic. I took a deep breath, watched the site logs like a hawk, and hit my “restore database” button.

And… nothing. I didn’t know why, but it didn’t seem to work. I ran it (erm, pressed the button) again. This time, I got two terms! I didn’t know what was happening, but guessed that my (empty) Terms collection was getting to many requests from incoming users to process my function. Google Analytics told me there were 30-40 people on the site, and I needed to shut off their access to the DB so I could work on it.

Again, I went for the simplest solution. I redirected anyone on the site to a simple white page that succinctly stated “down for maintenance”.  No DB access. Then I set my routes so that only the user named “Max” (erm, me) could bypass the page.

I pushed the code, navigated to my button, took a deep breath, and hit the Jesus button. I refreshed the Analytics page frantically to see how many terms I had. Within a minute, the number climbed to 375. My fix worked flawlessly. I tested the site, and definitions showed up just fine. I was back.

I turned off maintenance mode, and users came flooding back from Reddit. I had been down for an hour. Nobody cared; nobody knew. The users who saw Hackterms down probably left and never came back, but there were hundreds of other users behind them. I braced myself and checked the threads. It was only 11AM (NYC time) in the morning, and the SF crowd was just waking up, and the NY crowd was just getting to brunch. I knew the bulk of my users would be visiting later, when the site was running.

Needless to say, I didn’t touch the definitions for the rest of the day.


 

r/Programming loved Hackterms, and I was on top of the subreddit once again. Visitor number hovered between 80-100, and the site wasn’t crashing!

Screen Shot 2018-03-03 at 5.16.47 AM.png

I was getting lots of positive feedback, and users were adding terms from my “most searched” list, which was a nice positive – the feature was appealing to users, and more importantly, guided eager new contributors. I continued to invite the most passionate contributors to a Slack channel, which gave me an opportunity to run ideas by (and gather ideas from) a group of dedicated followers.

Over the next few weeks, I obsessively checked Twitter for mentions of Hackterms, responded to every single user who signed up, and kept stirring the pot in the Slack community, getting some great feedback. I also became more active on Indiehackers, offering advice where I could, looking out for similar products, and asking for advice in return. One user even generously drafted a Hackterms redesign (which I continued to tweak, with Esther’s help).

header comparrison

Redesigned header (old header – top, new header – bottom)

I added a number of new features based off the user feedback, including this changelog to help my users (and, as it turns out, me!) keep track of the changes. Most significantly, I added cross-definition linking, automatically connecting terms to other definition on the website – I got a brilliant solution from my ex-Codecademy colleague, Reed (“just let the browser do the work!”).  I also added support for markdown and redesigned the search page header.

Screen Shot 2018-04-15 at 1.34.34 PM

Automatic cross-term linking


Screenshot_20180402-082033_Chrome.jpgOn April 1st, I was browsing the web, looking at all the awesome April Fool’s pranks, and thought to myself – I could do this next year! And then I thought – why wait till next year? I have 7 users total, four of which are my friends – I can afford to take a risk! So, within a few hours, Cluckterms was born.

Somewhere around this mark, to my amazement, the site crossed 1000 terms (~1100 definitions). The last nearly 100 came from a particularly enthusiastic user, and I challenged the Slack community (and prodded my close friends) to help get us over the finish line. 1000 is a completely arbitrary number, but it sounded significant to potentially talk to reporters. “Yep, our website is at 1000 terms.”

I also applied to YC. This was a scary thought, but a pretty straightforward process, and one that I felt I owed to myself. For years, I idolized YC, loved (and worked at!) YC companies, read YC advice, and dreamt of one day building my own thing. And now, I was able to actually build (with my own knowledge) something – maybe not great – but something that attracted users, something I could apply with. If you’re curious, you can watch my video here.

I didn’t know if I’d get in, and I kept my expectations low. On one hand, I had users and a working prototype. On the other, I didn’t see a billion dollar market and haven’t made a dime. I just built a thing I would have wanted, and it was clear others wanted it too.

My product management apprenticeship at the NYC Mayor’s Office picked up, and I found it harder to motivate myself to work on the site some evenings, though I was challenging myself to write even a little code in the mornings and during lunch. Growth slowed a bit, and I once again found myself in a little bit of a slump. Looking back at this blog post helped – because I was in the same place before I launched on Reddit in ~December/January, sitting around with 25 users.

I now had over a thousand terms, a cool design, and proof of concept. People seemed to dig Hackterms, and even during a total slog, the site grew little by little each day.

Screen Shot 2018-04-15 at 2.09.43 PM

The slow days

I needed to expose the site to a wider audience, get some press, and (gasp) make even a dollar in donations/revenue. If there was a distant feature where my life would involve making products useful to people, than I needed to prove to myself that someone was willing to pay me for my work. I didn’t know the whole path forward, but I knew there were three big (and scary) launches I could target:

  • Product Hunt
  • Hacker News
  • r/dataisbeautiful

Each of there could take my site down technically, and with criticism – but I had to move forward somehow, regardless of what YC said (and frankly, I had no expectation of getting in).

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s