Dear Connector Experts,

for our Enterprise data integration project we have the need to use Apache Kafka / Confluent as the "data highway".

This requires to send data to PI DA/AF from Kafka and also to read from PI DA/AF back into Kafka.
I know that there are Integrators to actually read, but I also need the write-into-PI-option.

Any comment on this if you have any plans on this would be great.
The following data types are in focus for us for this:
- Process Data (time series data) of Sensors / Actuators
- Batch Data (Event Frames)
- AF Content (Plant structure information)
- data needed to automatically create Tags (optional)

Many thanks in advance!

Despite having released hundreds of PI Interfaces and dozens of PI Connectors, OSIsoft struggles to keep up with the ever-increasing number of data sources. Meanwhile, there's Power BI Desktop, a free program by Microsoft that allows users to combine and reshape data from different sources by using a simple query language (Power Query) or the even simpler GUI. It is a perfect match!

The PI Connector For Power BI Desktop would instantly give PI support for many more data sources, and users can always write custom data connectors for data sources that are not natively supported and possibly share them on GitHub. This effectively outsources and deduplicates a lot of the work related to connecting to different data sources. Maybe OSIsoft might even decide to create and share a custom data connector, which benefits both OSIsoft customers and Microsoft customers.

The PI Connector For Power BI Desktop would let users easily combine data from different sources and then save only the required values into PI, which saves tags, disk space, and processing power compared to saving the raw data into PI first and using the PI Analysis Service to derive new values.

The PI Connector For Power BI Desktop would replace some existing PI Connectors and satisfy some of the suggestions for new PI Connectors, giving users a much more unified and simplified experience. For example, since Power BI Desktop can handle both text sources and relational database sources, the PI Connector For Power BI Desktop would supersede most of the PI Connector For UFL and all of the idea of a PI Connector For RDBMS. The user can also skip learning SQL and the INI file language and focus on just Power Query, which is not only 1 language instead of 2, but also much easier to learn, read, and write and much more useful to the user outside of PI work. In addition, Power BI Desktop has a GUI that allows users to build queries without writing code and to preview and undo any step.

The only downsides that come to mind are that there may be licensing fees from Microsoft and that the PI Connector For Power BI Desktop would likely be too slow to collect data from sources that send data more often than every few seconds, so OSIsoft would still need to make PI Connectors for such sources. Despite this, a PI Connector for a program as solid as Power BI Desktop would undoubtedly benefit PI users in their data retrieval journey.

Summary:

Pros:
• Combine and reshape data like never before in a PI system
• Instant spike in the number of supported data sources
• Outsources and crowdsources the work for developing new connectors; a symbiotic relationship with Microsoft and its customers
• Can reduce the strain on the PI system
• Reduces the number of PI Connectors needed
• Reduces the number and difficulty of query languages to use; GUI can dodge coding entirely
• Knowledge of Power Query and Power BI Desktop is useful outside of PI work

Cons:
• Likely too slow for real-time data
• May have a licensing fee