Sequential Bayesian Learning

Learning unknown states as a sequence of beliefs, not as one-shot estimates.

Why Sequential Bayesian Learning?

Many scientific and engineering systems are only partially observed. A robot sees noisy sensor readings rather than its exact state; a medical monitor observes symptoms rather than the full health condition; a financial system observes prices rather than the hidden market regime; a wireless network observes pilots rather than the full channel. Sequential Bayesian learning provides a principled way to infer such hidden quantities as data arrive over time.

The key idea is to maintain a belief distribution, meaning a probability distribution over possible hidden states, rather than a single hard estimate. At time \(t\), the system has a prior belief formed from past data. A new observation arrives, the belief is updated by Bayes' rule, and the result becomes the prior for the next time step. This structure appears in hidden Markov models (HMMs), Kalman filters, particle filters, Bayesian smoothing, variational inference, and modern probabilistic machine learning.

Wireless communication is one application domain used in the reference papers in this folder. In that setting, HMMs, switching Kalman filtering (SKF), trajectory inference, covariance estimation, and spatial regularization are used to learn radio maps and track channel states from sequential measurements. The same theoretical structure can also support robotics, target tracking, environmental sensing, predictive maintenance, biomedical monitoring, and time-series forecasting.

The Core Idea

A convenient abstraction is a state-space model:

p(z_t \mid y_{1:t}) \propto p(y_t \mid z_t)\int p(z_t\mid z_{t-1})p(z_{t-1}\mid y_{1:t-1})\,dz_{t-1}.

Here, \(z_t\) is the hidden state at time \(t\), \(y_{1:t}\) is the observation sequence up to time \(t\), \(p(y_t\mid z_t)\) is the observation model, and \(p(z_t\mid z_{t-1})\) is the state-transition model. The integral sums over all possible previous states. This equation says: predict the next state using the transition model, then correct that prediction using the new observation.

As a wireless application example, \(z_t\) may contain a user's location, channel state, line-of-sight/non-line-of-sight (LOS/NLOS) condition, or a map-related parameter. Line-of-sight means that the direct propagation path is not blocked; non-line-of-sight means that the direct path is blocked and the signal arrives through reflection, diffraction, or scattering. A radio-map construction problem without location labels can then be written as a maximum-likelihood trajectory problem:

\widehat{X}_{1:T}=\arg\max_{X_{1:T}}\Big[\log p(Y_{1:T},X_{1:T})+\lambda R(X_{1:T},Y_{1:T})\Big],

where \(X_{1:T}\) is the unknown trajectory, \(Y_{1:T}\) is the observed channel state information (CSI) sequence, \(R(\cdot)\) is a spatial regularization term derived from CSI distance or propagation structure, and \(\lambda\) controls how strongly the spatial regularizer is trusted. CSI describes the measured or estimated wireless channel response between transmitter and receiver.

The same mathematical template can be reused in many other domains. In robotics, \(X_{1:T}\) could be a robot trajectory inferred from camera or lidar data. In environmental monitoring, it could be a hidden pollution field inferred from sensor readings. In biomedical signal analysis, it could be a latent physiological state inferred from wearable measurements.

Models at a Glance

Sequential Bayesian model

Figure 1. A sequential Bayesian model updates hidden states from sparse observations while prior structural knowledge constrains the inference. While the CSI observation \(\boldsymbol{y}_t\) may have a very high dimension and may appear to vary "randomly", the corresponding physical location of the mobile user travels in a two-dimensional (2D) space with a continuous trajectory \(\mathbf{v}_t\). Such trajectory may be discovered by sequential Bayesian learning. Here, \(P_{ij}\) is the Markovian transition probability to be learned by the model.

Bayesian bandit beamforming

Figure 2. A wireless application example from Bayesian bandit beamforming: The system bypasses a conventional channel estimation/training stage. Instead, it directly steers the beam towards the user for data transmission without knowing the exact CSI. By adaptively adjusting the beam in a sequential way with feedback from the user, the beam can quickly converge to a desired one (usually in 10-20 steps for a system with 100 antenna elements, depending on the channel sparsity [4]). This is important in a system with reconfigurable intelligent surface (RIS), because the RIS beam can be either optimized for pilot transmission (with a poor data payload capability) or optimized for data transmission (with a poor channel sensing capability).

A Typical Workflow

Step 1: Collect sequential observations

The input is an ordered stream of observations \(y_1,y_2,\ldots,y_T\). These observations may be sensor readings, images, received signal strength/channel state information (RSS/CSI) measurements, biomedical signals, industrial measurements, or financial time series. The important point is that time order carries information about the hidden process.

Step 2: Infer hidden states

Choose a transition model and an observation model, then update the posterior distribution over the hidden state. A posterior distribution is the updated probability distribution after the new observation is included. Depending on the model class, this can be done with an HMM, a Kalman filter (KF), a switching Kalman filter (SKF), a particle filter, an expectation-maximization method, a variational approximation, a message-passing algorithm, or a deep learning approach may be used.

Step 3: Use the inferred structure

The inferred state sequence can support prediction, control, map construction, anomaly detection, resource allocation, or decision-making. In wireless applications, the recovered hidden states can support CSI tracking, beam prediction, covariance estimation, and radio-map updates without requiring a costly new labeled measurement campaign.

Theoretical Foundations: Hidden Geometry in Sequential Data

References [1]-[4] in this section refer to the works listed under Ours in the suggested reading.

1. CRLB results explain when trajectory recovery is possible

The blind angular-power-map paper [2] studies whether location can be recovered from unlabeled CSI sequences. Under uniform rectilinear mobility and Poisson-distributed base stations over a sufficiently large region, we found that the Cramer-Rao lower bound (CRLB), a theoretical lower bound on the variance of any unbiased estimator, can vanish asymptotically at any signal-to-noise ratio (SNR). The localization error decays as \(O(1/T)\), where \(T\) is the number of measurements. However, when base stations are restricted to a limited region, the lower bound does not vanish even with infinite measurements.

The lesson is structural: trajectory diversity and base-station geometry determine whether the hidden physical coordinates are identifiable.

2. Spatial regularization helps identifiability in NLOS

The indoor blind mapping paper [3] proves a spatial-continuity property for CSI under a quasi-specular NLOS environment and derives a CSI-distance metric proportional to physical distance under fully scattered NLOS conditions. A metric is a rule for measuring distance or dissimilarity. This makes it possible to regularize trajectory inference using channel similarity, rather than relying only on inaccurate angle estimates.

3. Bayesian bandits can achieve sublinear regret

The RIS Bayesian bandit paper [4] studies a sequential decision problem where each RIS phase pattern must both sense the channel and serve data transmission. The method uses Thompson sampling: at each slot, it samples a plausible cascaded channel from the current posterior distribution, chooses a phase pattern that would work well for that sampled channel, observes one scalar pilot feedback value, and then updates the posterior.

The key theoretical result is that the received-power regret is controlled by how fast the posterior uncertainty contracts. Regret means the cumulative performance loss relative to an oracle that already knows the true channel. The paper further proves a conditional sublinear Bayesian-regret guarantee: if the selected RIS patterns accumulate enough information over time, then the average regret \(R_T/T\) vanishes as the horizon \(T\) grows. This gives a useful theoretical message for sequential Bayesian learning: exploration is not just a heuristic; when the actions keep the posterior informative, adaptive decisions can become asymptotically efficient.

What Performance Advantage Do We Get?

The bracketed references in this section refer to Ours in the suggested reading.

Scenario	What the method uses	Reported advantage
Reduced-pilot massive multiple-input multiple-output (MIMO) CSI tracking [1]	Radio-map-embedded SKF with adaptive sensing: first discover the proxy location, and then user location to assist for CSI prediction	In the ray-traced 64-antenna multiple-input single-output (MISO) setting reported in [1], using 1 pilot observation every 10 ms achieved over 97% of perfect-CSI capacity, while the conventional KF baseline achieved 76%.
Blind radio-map construction for CSI tracking [1]	HMM-based joint location and covariance estimation	In the radio-map construction experiment in [1], the proposed model reduced localization error from a 30 m prior to about 6 m just based on a narrowband channel without ToA information.
Blind angular power maps [2]	HMM trajectory recovery from large-timescale CSI	In the commercial fifth-generation (5G) massive MIMO dataset in [2], the method reported mean localization error below 18 m despite many neighboring-cell measurements being missing.
Spatially regularized indoor mapping [3]	LOS/NLOS inference plus CSI-distance regularization	In the ray-tracing indoor MIMO orthogonal frequency-division multiplexing (OFDM) experiment in [3], the method reported 0.68 m average localization error, 2% LOS/NLOS identification error, and 3.3% relative MIMO beam-map error.

These values are tied to the experimental scenarios and baselines in the cited papers. They should be read as evidence that sequential structure is valuable under the studied conditions, not as universal guarantees.

Why This Research Matters

Sequential Bayesian learning is a broad theoretical language for making decisions from incomplete, noisy, and time-ordered data. The wireless examples in this page show one concrete use case, but the same ideas are much wider.

For theoretical research, sequential Bayesian learning is a foundation for inference in partially observed dynamical systems. It connects HMMs, Kalman filtering, Bayesian smoothing, expectation-maximization, variational learning, particle methods, and deep generative models under one probabilistic viewpoint.

For real industrial applications, sequential Bayesian learning supports tracking, forecasting, anomaly detection, map construction, resource allocation, and control in systems where direct observation is incomplete or noisy. Wireless networks are one example; robotics, autonomous driving, industrial monitoring, finance, environmental sensing, and healthcare all use related models.

For prospective students who are interested in theoretical research, this topic offers a concrete path into probability, statistical signal processing, optimization, and learning theory. The mathematics is abstract enough to transfer across domains, but concrete enough to test on real sequential data.