5 steps to use logstash plugin for New Relic Logs

In this article, we will see 5 easy steps to install logstash plugin for New Relic logs. You don't need to go through the complete documentation of Logstash and New Relic. Just follow these simple steps and your job is done.

Let's say, our requirement is to forward logs from any server to New Relic Logs. New Relic Logs is a new kind of log monitoring solution. You can strip away the complexities of cumbersome on-prem log management and boost your business results with a cost-effective, cloud-based solution.If you would like to read more about New Relic Logs, Please use this link. Documentation is available here.

New Relic Logs provides different ways to bring logs into New Relic. You can use following 3 options.

We are going to see the log forwarding plugin in this article. New Relic supports many log forwarders. Let us take logstash and go over the 5 easy steps to install logstash and forward logs to New Relic Logs.

Install Java
Install Logstash
Install New Relic Plugin for Logstash
Configure New Relic Plugin
Validate the installation

1. Install Java:

One of the pre-requisites for Logstash is to have correct version of Java installed. Logstash requires one of the java verions Java8 , Java11, Java14. I downloaded Java14 from Oracle site and installed using the following command.

NOTE : If you are installing java to any custom location, you need to link the java to your new location as below.

2. Install Logstash:

Logstash is installed using rpm. There are various methods to install logstash. We used rpm method for this setup. Please complete the following steps to install logstash.

3. Install New Relic Plugin for Logstash :

Use the following steps to install New Relic Plugin for Logstash.

Verify if Plugin is installed. Go to directory /usr/share/logstash

Install plugin using....

Validate again if the plugin is installed....

4. Configure New Relic Plugin:

At this point, we have logstash installed and New Relic plugin for Logs is also installed. Now lets configure the plugin to forward logs to New Relic.

Above configuration is a simple configuration to send the logs to standard output. The content of the file specified for path will be sent to standard output.

5. Validate the installation:

We have successfully done the installation of Logstash and New Relic Plugin. And also configured the plugin. We can now test the plugin function. As mentioned in the section above, we are sending the content of the logfile to standard output for validation. Once the validation is done, we can configure the actual configuration files to direct logs to New Relic Logs.

NOTE: At this point, the logstash is not started. When you install logstash using rpm method, it will not start automatically. But there is a OS service that will be created during installation. Please read the next section to know how to start logstash.

For the validation, Start the logstash using the configuration file that we have created "/u001/app/ag_software/askm.log"

/usr/share/logstash/bin/logstash -f /u001/app/ag_software/askm.log

NOTE : Above command will not return the prompt.

Now we can populate the logfile with some content using the following command. Execute the below command from a different terminal.

Then you should expect the following output. If you see the following output, the configuration is successful.

Validating with New Relic Logs :

We have successfully installed Logstash plugin and New Relic Plugin and then validated Logstash plugin to send logs to standard output. We have not tested sending logs to New Relic Logs. In order to send the logs New Relic Logs,

Create the configuration file in location /etc/logstash/logstash.conf
Start the logstash service which is created as part of installation.
Send some text messages to the logfile configured in above configuration for path variable.
Validate the logs in New Relic.

askMLabs