On learning what to learn: Heterogeneous observations of dynamics and establishing possibly causal relations among them.

Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the and of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding "the right quantities" to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple of the system. We determine (i) what subsets of observables are between the observation processes (and therefore observable from each other, relatable through a function); and (ii) what information is to these common observables, therefore particular to each observation process, and not contributing to the desired function. Any data-driven technique can subsequently be used to learn the input-output relation-from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular "twists" of the approach are discussed. The first has to do with the of particular quantities of interest from the measurements. We now construct mappings from set of observations from one process to of measurements of the second process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures "now," while the second observation process measures "in the future," the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution.