next up previous
Next: Example: Southern oscillation index Up: Fourier based surrogates Previous: Flatness bias of AAFT

Iteratively refined surrogates

  In Ref. [30], we propose a method which iteratively corrects deviations in spectrum and distribution from the goal set by the measured data. In an alternating fashion, the surrogate is filtered towards the correct Fourier amplitudes and rank-ordered to the correct distribution.

Let tex2html_wrap_inline2008 be the Fourier amplitudes, Eq.(7), of the data and tex2html_wrap_inline2010 a copy of the data sorted by magnitude in ascending order. At each iteration stage (i), we have a sequence tex2html_wrap_inline2014 that has the correct distribution (coincides with tex2html_wrap_inline2010 when sorted), and a sequence tex2html_wrap_inline2018 that has the correct Fourier amplitudes given by tex2html_wrap_inline2008. One can start with tex2html_wrap_inline2022 being either an AAFT surrogate, or simply a random shuffle of the data.

The step tex2html_wrap_inline2024 is a very crude ``filter'' in the Fourier domain: The Fourier amplitudes are simply replaced by the desired ones. First, take the (discrete) Fourier transform of tex2html_wrap_inline2014:
equation1038
Then transform back, replacing the actual amplitudes by the desired ones, but keeping the phases tex2html_wrap_inline2028:
 equation1040
The step tex2html_wrap_inline2030 proceeds by rank ordering:
 equation1042
It can be heuristically understood that the iteration scheme is attracted to a fixed point tex2html_wrap_inline2032 for large (i). Since the minimal possible change equals to the smallest nonzero difference tex2html_wrap_inline2036 and is therefore finite for finite N, the fixed point is reached after a finite number of iterations. The remaining discrepancy between tex2html_wrap_inline2040 and tex2html_wrap_inline2042 can be taken as a measure of the accuracy of the method. Whether the residual bias in tex2html_wrap_inline2040 or tex2html_wrap_inline2042 is more tolerable depends on the data and the nonlinearity measure to be used. For coarsely digitised data,gif deviations from the discrete distribution can lead to spurious results whence tex2html_wrap_inline2040 is the safer choice. If linear correlations are dominant, tex2html_wrap_inline2042 can be more suitable.

The final accuracy that can be reached depends on the size and structure of the data and is generally sufficient for hypothesis testing. In all the cases we have studied so far, we have observed a substantial improvement over the standard AAFT approach. Convergence properties are also discussed in [30]. In Sec. 5.5 below, we will say more about the remaining inaccuracies.


next up previous
Next: Example: Southern oscillation index Up: Fourier based surrogates Previous: Flatness bias of AAFT

Thomas Schreiber
Mon Aug 30 17:31:48 CEST 1999