If you are interested to learn about the Ansible Java
For aspiring DevOps engineers, it is essential to know shell Scripting or bash scripting. In this shell scripting for DevOps guide, I will share my tips & resources to learn Linux shell scripting the right way.
I have spoken about the importance of shell scripting in my becoming a DevOps engineers guide as well. This guide is for anyone who wants to learn shell scripting the right way that can help you with your day-to-day work and DevOps interviews. It’s very important to have patience and discipline in the learning process.
Shell Scripting for DevOps
The first question you might have is, how important is shell scripting for a DevOps engineer? I have got this question from freshers as well experienced IT folks. The answer is, it’s very important. The following image shows the Stackover flow 2021 survey, where 27% of the respondents said they use shell scripting.
But most of the time, I get asked, saying that there are many automation tools out there and do we still need to learn and write shell scripts? The answer is yes. We no more create shell scripts for full-fledged automation but we use scripting along with automation tools and for ad-hoc tasks.
For example, if you use the AWS user data option, there is a chance you will use a shell script inside it. Another example is, while creating AMI images using a packer, you will end up using a shell script for the AMI configuration. Also, I have had instances where I had to use shell scripts with configuration management tools, containers, etc. We will talk more about this in the following sections. Also, shell script comes in handy for repeatable development tasks. For example, it could be spinning up a Vagrant VM with the essential software or setting up the development environment itself.
Most importantly, hands-on scripting and programming are becoming mandatory in preliminary interview rounds in DevOps interviews. So it is another crucial reason why you should learn shell scripting.
How to Get Started With Shell Scripting?
The prerequisite to getting started with shell scripting is hands-on experience with Linux. Therefore, before getting started with shell scripting, you should be comfortable working with Linux commands.
If you are totally new to the Linux world, you could start with spinning up a few Linux servers locally using Vagrant or on cloud platforms like AWS, Google Cloud, etc. Every platform offers free credits for beginners. You can check out the free $100 credit from the digital ocean as well. It’s effortless to get started. If you think you are comfortable with Linux, you can get started with Shell scripting basic tutorials. Also, I want you to do the following.
Create a Github repo, create folders for each concept you learn, and commit all your work scripts. It doesn’t matter if the script is available online. Have your own documentation. You can add references and credit websites/blog links in the README section. Once your learning is over, you can share it on Linkedin. Trust me; it will help you retain the knowledge and will help others as well.
What are the Best Free Shell Scripting Learning Resources?
You don’t need to pay for any expensive courses to learn Linux shell scripting. There is too much marketing around courses that promises to make you a DevOps Shell scripting ninja. However, I don’t have anything against courses; go for it if you feel comfortable learning via a paid course. I will add those resources as well.
I have listed down the best free shell scripting resources for you to get started. Go through all the resources once, pick one resource you feel comfortable with, and ensure you go through the material from start to finish. As I said earlier, mastering technology requires patience and discipline.
Real-world Shell Scripting Use Cases
Assuming you have learned all the shell scripting concepts and probably written and executed shell scrips for learning purposes. The next question anyone would have is, how to learn real-world shell scripting use cases? If you work in a company, you can find project documentation and infrastructure automation codes where existing projects use shell scripts. If you don’t have access to project documents and code, here is what you can do. Find the Github repo of the most common docker base images, For example, the Nginx base image. There you will find an entrypoint folder with multiple scripts or an entrypoint.sh script file. Here is an nginx example
Try to understand the shell script and its functionality. In this process, you will figure out many concepts and logic that can be used while writing a shell script. (functions, conditionals, switch cases, and many more). It is not just limited to Docker repositories, you can find scripts in many open source community repositories.
Shell Scripting Real Time Scenarios
It is always better to learn with some real-time scenarios in terms of DevOps engineer interviews. So I have come up with some real-world scenarios you can try out. I will keep updating this list.
- Write a shell script function to find and kill all the zombie process.
- Find the first 10 biggest file in the file system and write the output to a file.
- Shell script to gracefully unmount a disk.
- shell script to send email
- Shell script to monitor CPU, Memory and Disk usage and send the output to a file in table format.
- Shell script to find the files created and its sizes. It should accept the number of days as input. Or a from and to date format as inputs.
- Write a shell script to list of users logged in by date and write it to an output file.
- Shell script to Copy files recursively to remote hosts
- Shell script that displays the number of failed login attempts by IP address and location.
- Shell script to parse a log file and forward a specific value with a timestamp to an output file.
Ensure you include the following concepts in the shell scripts you are writing.
- variable definitions
- Use cut, awk and grep
- Handling Input/output /error redirections.
- Conditions/If else statements
- Case statement scripts
- Loops (For/Do-While)
- Exist status
- Command line arguments
Shell Scripting Languages
First of all, there are no specific shell scripting languages. When it comes to shell scripting, it refers to sh, bash, csh, tcsh. Here we are primarily talking about bash shell scripting. Bash is an interpreted scripting language for Unix-based systems.
A shell script is a computer program designed to be run by the Unix shell, a command-line interpreter.
Wikipedia
But scripting languages, in general, means multiple languages. The following are the common scripting languages.
- Powershell
- Python
- Pearl
- Groovy
Shell Scripting DevOps Interview Questions
Shell scripting DevOps interview questions differ from company to company. For example, a Service based company would be just interested in your basic Linux and shell scripting knowledge. However, a product-based company might expect a good level of knowledge in Linux command line and shell scripting. Nowadays, most companies would provide platforms like Hackerank to test your scripting skills. So it is necessary to practice shell scripting to solve problems.
Following are some of the DevOps interview shell script questions.
- Shell script to reverse a string
- Parse a log in desired format and find all occurrence of a word
- Schell script to send the first ten lines of a file to another file.
- Merge(contatinate) two files and write the output to the third file.
- Find all duplicate strings in a file and replace it with another string
- Find all the IP adress from a log file and write it to another file.
I will be updating the list with more generic interview questions soon.
Shell Scripting FAQ’s
Following are some of the frequently asked shell scripting questions by aspiring DevOps engineers.
Why shell scripting is needed in DevOps?
With shell scripting, you can automate the tasks which consume a lot of time. Shell scripting is a way for you to save a lot of time and focus on the things that matter rather than wasting time on unnecessary time consuming repetitive tasks
Which scripting is required for DevOps?
Python: Python is one of the best choices for DevOps operations. It is mostly used as a coding language and is more user-friendly than any other languages of the similar level.
How hard is it to learn Linux shell scripting?
Learning shell scripting is like learning any other technology. If you have a good foundational knowledge of Linux, you can learn shell scripting easily, provided you follow a disciplined learning path.
Is scripting required for DevOps Engineers?
Yes. Scripting knowledge is very essential for a DevOps engineer. For the development environment till production, a DevOps engineer might have to create useful utilities using scripting.
Which scripting language is best for DevOps?
It depends on the project. But Bash, Powershell, and python are the commonly used scripting languages in DevOps-based projects. Groovy is essential if the project uses Jenkins.
Is Shell Scripting used for automation?
With the advent of many DevOps tools, there is no need to write full-fledged automation in shell scripts. However, shell scripts are used as part of those automation tools to support a few functionalities. A classic example would be using shell scripting with Hasicorp Packer to build VM images.
