Thoughts on Pipes and Filters pattern

Any UNIX user is no stranger to the Pipes and Filters pattern. In fact, I consider it a signature of UNIX for its ease in combing various utilities (filters) to create a final output by piping the output of one utility as an input of another. Since this pattern enables the intermediate filters to execute in parallel, as the prevalence of multi-core systems increase, I can see it gaining even higher popularity.

The author lists direct function calling as the most simplistic but yet flexibility-limiting piping scheme. One variation I have seen to preserve flexibility, especially for passive filters, is to have a meta structure listing the ordering of the filters. When one filter produces output, it simply relies on the main routine or a controller to deliver it to the next filter based on the meta structure. To support a different pipeline configuration, one only needs to edit the meta structure.

Another variation that I've seen with uniform-data-type pipelines is instead of setting up various pipelines based on the current configuration, there is a super pipeline consisting of all filters. Then, based on the current configuration, a filter can be turned ON or OFF. When it is OFF, it simply pass along the result to the next filter.