TimeSeries趋势数据的重采样，聚合和插值

在分析能源需求和消费数据时，我有问题重新采样和插值时间序列趋势数据。

数据集示例：

timestamp value kWh ------------------ --------- 12/19/2011 5:43:21 PM 79178 12/19/2011 5:58:21 PM 79179.88 12/19/2011 6:13:21 PM 79182.13 12/19/2011 6:28:21 PM 79183.88 12/19/2011 6:43:21 PM 79185.63 

根据这些观察结果，我希望根据一段时间对某些汇总值进行汇总，将频率设置为一个时间单位。

如在，小时间隔填补缺失数据的任何空白

 timestamp value (approx) ------------------ --------- 12/19/2011 5:00:00 PM 79173 12/19/2011 6:00:00 PM 79179 12/19/2011 7:00:00 PM 79186 

对于线性算法，似乎我会在时间上采用差异并将该值乘以该因子。

 TimeSpan ts = current - previous; Double factor = ts.TotalMinutes / period; 

可以基于因子来计算值和时间戳。

有了这么多的可用信息，我不确定为什么很难找到最优雅的方法。

也许首先，是否有推荐的开源分析库？

对程序化方法的任何建议？ 理想情况下是C＃，还是可能是SQL？

或者，我可以指出任何类似的问题（答案）？

Answer 1

通过使用内部用于表示DateTimes的时间标记，您可以获得最准确的值。 由于这些时间刻度不会在午夜重新开始，因此您在日间边界不会出现问题。

 // Sample times and full hour DateTime lastSampleTimeBeforeFullHour = new DateTime(2011, 12, 19, 17, 58, 21); DateTime firstSampleTimeAfterFullHour = new DateTime(2011, 12, 19, 18, 13, 21); DateTime fullHour = new DateTime(2011, 12, 19, 18, 00, 00); // Times as ticks (most accurate time unit) long t0 = lastSampleTimeBeforeFullHour.Ticks; long t1 = firstSampleTimeAfterFullHour.Ticks; long tf = fullHour.Ticks; // Energy samples double e0 = 79179.88; // kWh before full hour double e1 = 79182.13; // kWh after full hour double ef; // interpolated energy at full hour ef = e0 + (tf - t0) * (e1 - e0) / (t1 - t0); // ==> 79180.1275 kWh 

公式的说明
在几何中，类似的三角形是具有相同形状但不同尺寸的三角形。 上面的公式基于以下事实：一个三角形中任何两个边的比率对于相似三角形的相应边是相同的。

如果你有一个三角形ABC和一个类似的三角形abc，那么A : B = a : b 。 两个比率的平等称为比例。

我们可以将这个比例规则应用于我们的问题：

 (e1 – e0) / (t1 – t0) = (ef – e0) / (tf – t0) --- large triangle -- --- small triangle -- 

Answer 2

我编写了一个LINQ函数来插值和规范化时间序列数据，以便可以聚合/合并。

重采样function如下。 我在Code Project上写了一篇关于这种技术的简短文章 。

 // The function is an extension method, so it must be defined in a static class. public static class ResampleExt { // Resample an input time series and create a new time series between two // particular dates sampled at a specified time interval. public static IEnumerable Resample( // Input time series to be resampled. this IEnumerable source, // Start date of the new time series. DateTime startDate, // Date at which the new time series will have ended. DateTime endDate, // The time interval between samples. TimeSpan resampleInterval, // Function that selects a date/time value from an input data point. Func dateSelector, // Interpolation function that produces a new interpolated data point // at a particular time between two input data points. Func interpolator ) { // ... argument checking omitted ... // // Manually enumerate the input time series... // This is manual because the first data point must be treated specially. // var e = source.GetEnumerator(); if (e.MoveNext()) { // Initialize working date to the start date, this variable will be used to // walk forward in time towards the end date. var workingDate = startDate; // Extract the first data point from the input time series. var firstDataPoint = e.Current; // Extract the first data point's date using the date selector. var firstDate = dateSelector(firstDataPoint); // Loop forward in time until we reach either the date of the first // data point or the end date, which ever comes first. while (workingDate < endDate && workingDate <= firstDate) { // Until we reach the date of the first data point, // use the interpolation function to generate an output // data point from the first data point. yield return interpolator(workingDate, firstDataPoint, firstDataPoint, 0); // Walk forward in time by the specified time period. workingDate += resampleInterval; } // // Setup current data point... we will now loop over input data points and // interpolate between the current and next data points. // var curDataPoint = firstDataPoint; var curDate = firstDate; // // After we have reached the first data point, loop over remaining input data points until // either the input data points have been exhausted or we have reached the end date. // while (workingDate < endDate && e.MoveNext()) { // Extract the next data point from the input time series. var nextDataPoint = e.Current; // Extract the next data point's date using the data selector. var nextDate = dateSelector(nextDataPoint); // Calculate the time span between the dates of the current and next data points. var timeSpan = nextDate - firstDate; // Loop forward in time until wwe have moved beyond the date of the next data point. while (workingDate <= endDate && workingDate < nextDate) { // The time span from the current date to the working date. var curTimeSpan = workingDate - curDate; // The time between the dates as a percentage (a 0-1 value). var timePct = curTimeSpan.TotalSeconds / timeSpan.TotalSeconds; // Interpolate an output data point at the particular time between // the current and next data points. yield return interpolator(workingDate, curDataPoint, nextDataPoint, timePct); // Walk forward in time by the specified time period. workingDate += resampleInterval; } // Swap the next data point into the current data point so we can move on and continue // the interpolation with each subsqeuent data point assuming the role of // 'next data point' in the next iteration of this loop. curDataPoint = nextDataPoint; curDate = nextDate; } // Finally loop forward in time until we reach the end date. while (workingDate < endDate) { // Interpolate an output data point generated from the last data point. yield return interpolator(workingDate, curDataPoint, curDataPoint, 1); // Walk forward in time by the specified time period. workingDate += resampleInterval; } } } } 

Answer 3

Maby是这样的：

 SELECT DATE_FORMAT('%Y-%m-%d %H', timestamp) as day_hour, AVG(value) as aprox FROM table GROUP BY day_hour 

你使用什么数据库引擎？

Answer 4

对于你正在做的事情，似乎你正在为初学者错误地声明TimeSpan ts =（TimeSpan）（当前 – 之前）; 还要确保当前和以前是DateTime类型。

如果你想看看计算或累积，我会看看TotalHours（）这里有一个例子，你可以看一个想法，如果你喜欢这里检查LastWrite / Modified时间是否在24小时内

 if (((TimeSpan)(DateTime.Now - fiUpdateFileFile.LastWriteTime)).TotalHours < 24){} 

我知道这与你的情况有所不同，但你会对如何使用TotalHours有所了解