This document explains how to modify the logic in streaming pipelines within RisingWave. Understanding these mechanisms is essential for effectively managing your data processing workflows.
NULL
for existing records.
cust_sales
:
cust_sales_new
with the new column sales_count
:
cust_sales
and rename cust_sales_new
to cust_sales
:
CREATE SINK ... FROM ...
statement, you have the option to specify without_backfill = true
to exclude existing data.
orders
is not a table but another materialized view, derived from tables order_items
and price
.
sales_count
to cust_sales
, we need to create the new materialized views cust_sales_new
and orders_new
first:
cust_sales
and rename cust_sales_new
to cust_sales
:
adult_users
that tracks the number of users aged ≥ 18.
age >= 18
to age >= 16
as a straightforward solution. However, this is not feasible in stream processing since records with ages between 16 and 18 have already been filtered out. Therefore, the only option to restore the missing data is to recompute the entire stream from the beginning.
Therefore, we recommend persistently storing the source data in a long-term storage solution, such as a RisingWave table. This allows for the recomputation of the materialized view when altering the logic becomes necessary.