Apache NiFi Course Cover Image

Apache NiFi ExecuteStreamCommand Processor

Get any of the courses at very special price. Offer is available only for limited time, so enroll quickly.

Apache NiFi – Beginners to Advance Guide: https://itpanther.com/apachenifi
Apache NiFi – Admin Guide – Know Clustering and Other Advance Topics: https://itpanther.com/apachenifiadmin
Liferay – Create Your Website Without The Need to Code: https://itpanther.com/liferay
Grafana – Learn to do Analytics with Grafana: https://itpanther.com/grafana
Redis – Learn World’s Fastest Database: https://itpanther.com/redis
BMC Control-M for Beginners: https://itpanther.com/controlm

NiFi’s ExecuteStreamCommand Processor executes an external command (can be a shell script, python script or java script etc) on the contents of a flow file, and creates a new flow file with the results of the command.

Many a times, we may end up in a scenario when existing NiFi processors are not enough to handle the custom logic which we have. In situations like these we’re left with one of below two options:

  1. Create a custom NiFi Processor. This can be tough if you don’t know Java, as NiFi processors are written in Java.
  2. Write a script in our choice or programming language and then call this script using NiFi

So in this article we will learn about the second approach, calling a custom script (python script) using Apache NiFi ExecuteStreamCommand Processor.

You can watch the complete video here, If you like to read, then continue scrolling down.

Running a Custom python Script To Process a Flowfile – ExecuteStreamCommand

Step 1: Use ExecuteStreamCommand Processor

Step 2: Right-Click on ExecuteStreamCommand Processor and go to “Configure”


Step 3: In “Command Path” property, Provide the full path of the script. In the example below, it is assumed that there is “check_duplicate.py” python script exists at “/home/itpanther” directory.

Step 4: If your script is accepting any arguments, you can provide that in “Command Arguments” property. If your program accepts multiple arguments, then you can pass multiple arguments by separating them with semicolon.

Step 5: Please note that ExecuteStreamCommand processor passes the flow file content to system buffer, so your custom program should be able to read it from “stdin”. Each programming syntax has its own way to read from “stdin”.

Step 6: Once you process the data within your script, you can return the output back by sending the output to “stdout”.

Get any of the courses at very special price. Offer is available only for limited time, so enroll quickly.

Apache NiFi – Beginners to Advance Guide: https://itpanther.com/apachenifi
Apache NiFi – Admin Guide – Know Clustering and Other Advance Topics: https://itpanther.com/apachenifiadmin
Liferay – Create Your Website Without The Need to Code: https://itpanther.com/liferay
Grafana – Learn to do Analytics with Grafana: https://itpanther.com/grafana
Redis – Learn World’s Fastest Database: https://itpanther.com/redis
BMC Control-M for Beginners: https://itpanther.com/controlm

Leave a Comment

You have successfully subscribed to the newsletter

There was an error while trying to send your request. Please try again.

IT Panther will use the information you provide on this form to be in touch with you and to provide updates and marketing.
%d bloggers like this: