If we are working on production servers, we come across situation with performance issues like slowness in web server, particular service is not working, 5xx response codes to the users.
Below is the basic list of steps you can take to find the issue. We assume that atleast ssh is working in the server, if not we have physical access to the server.
- Check running services: a) services status-allb) topc) ps -a
Suppose if we are running web server, we need to check if web server like Apache or Nginx is running, dhcpd deamon is running, if database on the same server like MongoDB so check logs of database also with the database service is running or not.
If bind DNS configured, check if bind deamon is running. We can filter the process using “grep” command and use –i option to ignore case-sensitive search.
For example: $ps -A | grep -i “dhcp”
This will show only process running with dhcp. To know all options, use: $man <command name>
2. Check logs: Use /var/logs/syslogs to see all the system logs. Follow this link to see the directories for specific service logs.
Use head, tail and less options to view limited logs
There will be different logs for different services. For example: To check apache logs use: $cat /var/log/apache/*
3. Check Memory: (inodes and swap space) Its important to check if there is no memory issue.
We can also check memory, CPU related information from /proc directory.
For example: For
Memory: $cat /proc/meminfo
CPU: $cat /proc/cpuinfo
4. IPTABLES: Check if any service/Port or IP is blocked through IPTABLES.
5. Check disk size
du -sh * | sort -hr | head -n10
du -max /dir | sort -n
Check system logs to find any anomaly or errors
Check required services are running (HTTP, DNS, DHCP, NTP, Database)
IPTABLES are configured properly
NETSTAT command to check if connections are in waiting state.
Check CPU utilization, memory (physical, RAM and swap space)
dig from outside to the server, ping and traceroute (to find any issues in transit path)
For more detailed view of traffic, we can use TCP dump (commandline) or wireshark (GUI) to view the packets (incoming and outgoing) from the network interface.