set_bit

Applies to VAST Cluster 5.5.

Functions › Binary, hash and UUID functions › Bit strings. Returns a copy of a bit string with the bit at one index set to 0 or 1. Index 0 is the leftmost bit: set_bit('101101'::BIT, 1, 1) is '111101'.

Syntax

set_bit(bits, index, new_value)

Returns: BIT, the same length as bits.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_binary_hash_uuid_set_bit. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of five feature-flag sets, stored as VARCHAR strings of 0s and 1s. The third row has no values:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_binary_hash_uuid_set_bit"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.flags"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, bits VARCHAR)")
run(f"INSERT INTO {TABLE} VALUES "
    "(1, '1011'), "
    "(2, '0'), "
    "(3, NULL), "
    "(4, '10000000'), "
    "(5, '0110')")

[]
[]
[(5,)]

What it does

set_bit returns bits with the bit at index replaced by new_value. If the bit already has that value, the result is the same as the input. The input isn't changed: set_bit returns a new value.

The result is a BIT value, which the client receives as bytes in an internal encoding, so cast it to VARCHAR to read it. A string literal of 0s and 1s is cast to BIT, so set_bit('101101', 1, 1) works; a VARCHAR column must be cast: CAST(bits AS BIT).

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT set_bit(NULL, 0, 1), set_bit('101'::BIT, NULL, 1), set_bit('101'::BIT, 0, NULL), "
    "CAST(set_bit('101101'::BIT, 0, 0) AS VARCHAR), CAST(set_bit('101101'::BIT, 0, 1) AS VARCHAR)")
run("SELECT set_bit('101101'::BIT, 1, 2)")
run("SELECT set_bit('101101'::BIT, 6, 1)")
run("SELECT set_bit('101101'::BIT, 1, true)")

[(None, None, None, '001101', '101101')]
OSError: The new bit must be 1 or 0
OSError: bit index 6 out of valid range (0..5)
InternalError: Binder Error: No function matches the given name and argument types 'set_bit(BIT, INTEGER_LITERAL, BOOLEAN)'. You might need to add explicit type casts.

Examples

Turn on the first flag of each row. Rows 1 and 4 already have it:

run(f"SELECT id, bits, CAST(set_bit(CAST(bits AS BIT), 0, 1) AS VARCHAR) FROM {TABLE} ORDER BY id LIMIT 10")

[(1, '1011', '1011'), (2, '0', '1'), (3, None, None), (4, '10000000', '10000000'), (5, '0110', '1110')]

What the client receives without the cast to VARCHAR:

run("SELECT set_bit('101101'::BIT, 1, 1), CAST(set_bit('101101'::BIT, 1, 1) AS VARCHAR)")

[(b'\x02\xfd', '111101')]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). VAST DataBase can evaluate a comparison of a column with a constant, but not a function of a column, so set_bit() is always computed by the Query Engine:

• In the SELECT list, the scan reads the column and the Query Engine computes set_bit() for each row it returns.
• In WHERE, the scan sends the column for every row, and a FILTER step in the Query Engine computes set_bit() and keeps the rows that match. No rows are skipped in storage.

The plan of a filter on set_bit() shows the FILTER step above the scan:

plan(f"SELECT id FROM {TABLE} WHERE CAST(set_bit(CAST(bits AS BIT), 0, 1) AS VARCHAR) = '1011'")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #1            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│ (CAST(set_bit(CAST(bits AS│
│  BIT), 0, 1) AS VARCHAR) =│
│           '1011')         │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│            bits           │
│             id            │
│                           │
└───────────────────────────┘

Make it faster

• Filter with pushed-down conditions first. set_bit() in the SELECT list runs only on the rows the scan returns, so a condition on a raw column compared with a constant, such as id > 100, cuts the work: it's evaluated in VAST DataBase. The plan for such a condition has no FILTER step:

plan(f"SELECT CAST(set_bit(CAST(bits AS BIT), 0, 1) AS VARCHAR) FROM {TABLE} WHERE id > 3")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│ CAST(set_bit(CAST(bits AS │
│   BIT), 0, 1) AS VARCHAR) │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│     Projections: bits     │
│                           │
└───────────────────────────┘

Limitations and errors

• An index past the end fails the query. An index outside the bit string fails the statement with bit index ... out of valid range. Bring the values to one length first, with bitstring, or store bit strings of one length.
• The new value is an integer 0 or 1. Any other integer fails with The new bit must be 1 or 0, and a boolean fails with No function matches the given name and argument types.
• Bit strings only. A number, BLOB, date, boolean, list or VARCHAR column as bits fails with No function matches the given name and argument types. The error lists the one form set_bit accepts: set_bit(BIT, INTEGER, INTEGER).

run(f"SELECT set_bit(bits, 0, 1) FROM {TABLE} LIMIT 1")

InternalError: Binder Error: No function matches the given name and argument types 'set_bit(VARCHAR, INTEGER_LITERAL, INTEGER_LITERAL)'. You might need to add explicit type casts.

Similar functions

run("SELECT CAST(set_bit('1100'::BIT, 3, 1) AS VARCHAR), get_bit('1100'::BIT, 3), "
    "CAST('1100'::BIT | '0011'::BIT AS VARCHAR), CAST('1100'::BIT & '0111'::BIT AS VARCHAR)")

[('1101', 0, '1111', '0100')]

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• Binary, hash and UUID functions: all the bit string functions.
• BIT values and casts.
• Where filters run and pushdowns.