Theory of getting started for free is responsible for one of the most common mistakes companies make when choosing their log analytics solution. They forget overhead that will be created by the decision in later stage and what value they get compared to the other solutions.

Free ELK stack (Elasticsearch, Logstash, Kibana) is not as free as it is cracked up to be.

This post will focus on the costs of maintaining your own ELK stack and the alternatives to it.

So what are the costs of deploying ELK stack on AWS?

The answer to this question varies, and depends on several aspects like:

  • How much log data is generated by your system(s).
  • How long you want to retain that data.
  • How accessible your data has to be.

Lets take a case of a mid-size company having following general logging requirements:

  • 50GB of log data per day.
  • Retention period of 14 days.
  • High data availability.

Cost of hosted elastic search by AWS

1) 1 Master instance (c4.large, Asia Pacific, Mumbai):
You can check aws ec2 prices here
$0.10/hour * 720H/month = $72/month
2) 2 ES machines (r4.xlarge.elasticsearch):
You can check aws hosted elastic search pricing here
2 * $0.448/hour * 720H/month = $645/month
3) Hard Disk, EBS Standard volumes:
$0.162/hour * 50GB/day * 14/days retention * 2 (data redundancy) * 1.2 (recommended extra disk for ES) = $272/month
4) Monthly maintenance: About 2 days per month is the very least for this scale and it does not include crises (which do occur) and change requests from within the company: 530$/month.

Total estimated price for a simple managed ES on AWS with Kibana and Logstash: 
$72$ + $645$ + $272$ + $530 = $1519



Price for building your own ELK stack on AWS:

1) 1 Master instance (c4.large, Asia Pacific, Mumbai):
You can check aws ec2 prices here
$0.10/hour * 720H/month = $72/month
2) 2 data instances (r4.xlarge) according to ES recommendation + with must have redundancy:
$0.296/hour * 2 * 720 = 426$/month
3) Disk, general purpose SSD (gp2):
$0.12/hour * 50GB/day * 14/days retention * 2 (data redundancy) * 1.2 (recommended extra disk for ES) = 201$/month

3) Disk, general purpose SSD (gp2):

  • Setting up the entire stack including the ES servers, mapping, Kibana and collectors will take the average engineer which is familiar with the ELK stack about 5 working days which cost 530$/day according to the average daily salary of an engineer ($140K/year). Calculated monthly on a 2 years basis: 110$/month.
  • Monthly maintenance, about 3 days per month is the very least for this scale and it does not include crises (which do occur) and change requests from within the company: 1590$/month.

Total estimated price for a simple managed ES on AWS with Kibana and Logstash: 
$72$ + $645$ + $272$ + $530 = $1519