write_log

Applies to VAST Cluster 5.5.

Functions › Session and utility functions › Session and diagnostics. Writes a message to the Query Engine's log while the query runs, and returns NULL, or the value you pass as return_value. The message isn't sent to the client.

Syntax

write_log(message)
write_log(message, return_value := value)
write_log(message, log_type := 'type', return_value := value)
write_log(message, level := 'level', return_value := value)

The optional arguments are named, with :=.

Returns: the type of return_value; VARCHAR (always NULL) without it.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_string_write_log. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of three events. The last one has no message:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_string_write_log"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.events"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, msg VARCHAR)")
run(f"INSERT INTO {TABLE} VALUES (1, 'started'), (2, 'finished'), (3, NULL)")

[]
[]
[(3,)]

What it does

write_log(message) writes message to the Query Engine's log and returns NULL. With return_value := value, it returns value instead, with the type of value, so you can put the call where a value is expected. log_type := 'type' labels the entry. The query's results don't include the message.

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT write_log('abc'), typeof(write_log('abc')), write_log(NULL)")
run("SELECT write_log('abc', return_value := 42), typeof(write_log('abc', return_value := 42)), "
    "write_log('abc', log_type := 'docs', return_value := 1)")
run("SELECT write_log('abc', level := 'warning', return_value := 'x')")
run("SELECT write_log(123)")

[(None, 'VARCHAR', None)]
[(42, 'INTEGER', 1)]
InternalError: Not implemented Error: Enum value: unrecognized value "warning" for enum "LogLevel"
InternalError: Binder Error: No function matches the given name and argument types 'write_log(INTEGER_LITERAL)'. You might need to add explicit type casts.

Examples

Log each row's message as the query reads it. The column of results is NULL:

run(f"SELECT id, write_log(msg) FROM {TABLE} ORDER BY id LIMIT 10")

[(1, None), (2, None), (3, None)]

Log a message and return a value:

run("SELECT write_log('checkpoint reached', log_type := 'docs', return_value := 42)")

[(42,)]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). In the SELECT list, the scan reads the column and the Query Engine calls write_log() for each row it returns; the plan has no FILTER step:

plan(f"SELECT write_log(msg) FROM {TABLE}")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│       write_log(msg)      │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│      Projections: msg     │
│                           │
└───────────────────────────┘

Make it faster

• Log less. A call on a column runs for each row the query returns. Limit the rows with WHERE and LIMIT, or log a constant message once with a query of its own.

Limitations and errors

• The message must be a VARCHAR. A number fails with No function matches the given name and argument types; cast it to VARCHAR first.
• Not every level is accepted. level := 'warning' fails with Enum value: unrecognized value "warning" for enum "LogLevel"; the full message lists INFO as a candidate. Leave level out.
• The message isn't returned to the client. To show a message to the person running the query, return it as a column, or stop the query with error.

Similar functions

run("SELECT write_log('note', return_value := 'done')")
run("SELECT error('note')")

[('done',)]
OSError: note

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• error: stop a query with a message.
• Functions: all the function families.