Ingest data from PostgreSQL table

Describes how to ingest data from PostgreSQL table to RisingWave using table-valued function.

RisingWave allows you to query PostgreSQL tables directly with the postgres_query table-valued function (TVF). It offers a simpler alternative to Change Data Capture (CDC) when working with PostgreSQL data in RisingWave.

Unlike CDC, which continuously syncs data changes, this function lets you fetch data directly from PostgreSQL when needed. Therefore, this approach is ideal static or infrequently updated data, as it’s more resource-efficient than maintaining a constant CDC connection.

PUBLIC PREVIEW

This feature is currently in public preview, meaning it is nearing the final product but may not yet be fully stable. If you encounter any issues or have feedback, please reach out to us via our Slack channel. Your input is valuable in helping us improve this feature. For more details, see our Public Preview Feature List.

Syntax

Define postgres_query as follows:

postgres_query(
    hostname varchar,      -- Database hostname
    port varchar,          -- Database port
    username varchar,      -- Authentication username
    password varchar,      -- Authentication password
    database_name varchar, -- Target database name
    query varchar          -- SQL query to execute
)

Example

1. In your PostgreSQL database, create a table and populate it with sample data.

CREATE TABLE test (id bigint primary key, x int);
INSERT INTO test SELECT id, id::int FROM generate_series(1, 100) AS t(id);

2. In RisingWave, use postgres_query function to retrieve rows where id > 90.

SELECT * 
FROM postgres_query('localhost', '5432', 'postgres', 'postgres', 'mydb', 'SELECT * FROM test WHERE id > 90;');
----RESULT
91 91
92 92
93 93
94 94
95 95
96 96
97 97
98 98
99 99
100 100

Assistant

Responses are generated using AI and may contain mistakes.