8 Things to consider before setting up your Backend Platform

By Harsha TBA, VP Engineering (Backend) at Curofy

So you have just been hired in an early stage startup, or you just decided to startup! And your first task is to build a scalable backend that can support multiple apps, can do 1000 things, crunch all the required data, provide valuable insights and yes has a 0% downtime! Sounds familiar? If yes, then read on. Below are a few things responsible for its optimal functioning. The learnings are mostly centric towards the most common pitfalls that developers ignore. Most of these things are very low on effort compared to their usefulness.

1. Choosing the right language and framework

Choosing the right language and framework for your requirements can be a very confusing task. Start with the language you are comfortable with. Now while choosing a framework, first look at the size of the community using that framework. Second, an extensive ‘Documentation’ of the framework is also a must. Community and Documentation are very critical when you are building at pace.

If you are building an e-commerce site u can choose frameworks like Django, Rails, Magento which have some wonderful plugins to support your application. But if your application involves communication, choosing a non-blocking application framework is a must.

There has been a lot of talk going around Node vs other frameworks. Definitely, Node is the largest growing community but we need to give it some time till it matures enough.

2. Reusability

Developers are always under pressure to deliver fast and this might lead to bloated and unorganized code. Sorting out your code and breaking it into smaller functions can be real time saving. Moreover, you can write test cases for each function and profile them easily.

3. Never miss out on data

These days, the IP of any product is not the code or the design, it’s the data. And trust me you cannot afford to lose it.

So a couple of basic things...

(a) When storing a data row try storing additional information like the time of action. You never know when this will come useful.

(b) Never do hard deletion on any data. For example, if the user wants to delete some comment / purchase he made, do a soft deletion on that row i.e., have a flag in that row that determines that particular information has been deleted. This might be tough while writing the codes but this could provide very useful insights.

4. Never monolithic

There are a lot of cloud hosting services available which provide different prices and different features. But make it a point to not deploy all the services on just a single server. Instead, use multiple servers. This will not just provide load distribution but also prevents the case when the whole server crashes and you have to set everything up again.

In case you are using relational databases, AWS provides some out of the box solutions like RDS which have some amazing features like a point in time recovery and setting up read replicas.

5. One point deployments

Try to avoid ssh-ing to servers to deploy your code. There are many open source tools available like Chef, Ansible, and Fabric. You might be in your comfort zone in doing that but as u scale up this might prove to be real tough. Even when operating on a single server and installing some software try putting it in your deployment script. This will be really useful if you are rolling up a new server or replying your code when there is some mishap.

6. Version your repo and APIs

 Backward compatibility and maintaining the stable versions are sure herculean tasks while writing your code. Git and mercurial offer tagging your repositories. Try making a tag out of your repository over the time you make a deployment and figure out a versioning pattern. When something goes wrong you can immediately check out that tag.

Many things are going to change at the time of developing your code and you might realize a new API needs to be designed to optimize for that functionality (changing the format or data in the responses). Try maintaining different directories of your code for each version and route the requests based on the API version. This will ease your development and reduce the difficulty in making the APIs compatible each time a new feature is deployed.

7. Profile the performance

Performance sure is a necessity of any application written. As your code scales up, each module will become a black box and it will become tough to recognize where the performance is being deteriorated. You need to continuously monitor where the things are going wrong. There are great tools like New Relic which tell you the time taken by your queries, the queries which are taking too much time, and the ones you can optimize further.

8. Backup your Logs

Setting up log rotation and backing up your logs are generally ignored by any developer, not realizing the importance of it. There will be instances when you need to analyze which APIs are being called the most or what searches are being done on the app. Try setting up a different server to back up your databases and logs daily, or try plugging in free log parsing tools like Logstash. Alternatively, you can choose to use paid services like Loggly. This will make your job of searching through the logs really easy instead of going through a havoc of writing scripts in Sed and Awk.

This is definitely not an exhaustive list of everything you need to do to build a scalable backend. But these are the commonly ignored things that will cost you in near term only.