Scheduled backfilling for a specific time frame on analyses
Add the capability to perform scheduled backfills for a specific time frame, for analyses created on PI System Explorer.
Auto recalculation currently works for Expressions and Rollups, but not yet Event Frame generation analyses. Auto recalculation for EF generation analyses is in our backlog being prioritized.
Auto recalculation removes the need to schedule a backfill/recalc to support out of order inputs.
What are you using to enter in the manual data?
Mary N. Bellinger commented
Our case is manual data entry that is time-stamped when a sample is taken. The data entry may occur hours or days later, inserting late records to the PI Data Archive. When the data is off-spec, the original StartTime of the Event Frame describes the data entry timestamp instead of the timestamp of the data. The same is true of the EndTime. This can cause either fewer hours in excursion, or more hours than reality. We are backfilling the Event Frames to pick up the actual timestamps and recalculate the actual durations.
Looks to me there are two specific use cases here: 1) you have analyses that do not need to run more often than weekly - i.e. longer than what is permitted right now (daily), 2) you have data that is supposed to arrive at a specific time, but sometimes they don't and you want to re-calculate at a later time at which point you know for certain you have all the data you need for the calculations.
Did I get that right? Are there any more use cases?
Mohammad Reyaz Ali commented
I have business case where running more than 1000 analysis (required to run only on weekly basis) of keep running on real time which is not required and putting performance load on AF server.
Also same time analysis logic is dependent of some data which is not available in real time, which feed later before to run weekly schedule.
Stephen Flagg commented
As a system custodian, having this ability for the analytics team is essential to their effort.
David Koerkel commented
Actually, I think this enhancement would help me out as well. I receive data once a day between midnight and 4 AM from all of my electric meters (1.8 million) containing hourly and 15 minute data from the previous day midnight to midnight. when I take this data from each device and roll it up to it's parent transformer, the calculation is not quite right. I suspect I am getting interpolation. I have been working with support on this (case number 931664). If I do a backfil/recalc with the "delete existing data and recalculate" option selected, I get the expected summation to the parent transformer. Please reach out to me if you would like to discuss more details.
Gabriel Oliveira commented
I think it will be really helpful in cases where you need to filter data, e.g., while passing each filtered event to a new tag you need to know whether the following events within exactly 10 minutes are above a threshold.
Please describe use case in more details. Are you trying to handle late arriving data?
If you're trying to handle late arriving data, we're researching auto-recalculation for the 2017 R2 release.