bin

Applies to VAST Cluster 5.5.

Functions › Numeric and math functions › Number formatting. Returns an integer written in binary, as a VARCHAR of 0s and 1s: bin(5) is '101'. For a string, it writes the bits of each byte. to_binary is the same function.

Syntax

bin(x)
to_binary(x)

Returns: VARCHAR.

Aliases: to_binary.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_numeric_bin. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of four numbers and strings. The last row has no values:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_numeric_bin"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.nums"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, n BIGINT, s VARCHAR)")
run(f"INSERT INTO {TABLE} VALUES (1, 1536, 'hi'), (2, 5, 'A'), (3, -7, 'abc'), (4, NULL, NULL)")

[]
[]
[(4,)]

What it does

bin writes its argument in base 2:

Smaller integer types are widened to BIGINT first. That doesn't change a positive number (bin(5::TINYINT) is '101'), but a negative TINYINT, SMALLINT or INTEGER is written in 64 digits, not in its own size: bin((-1)::TINYINT) is 64 1s. Unsigned types are written as they are: bin(255::UTINYINT) is '11111111'. 0 is '0'.

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT bin(5), bin(0), bin(1), bin(NULL), typeof(bin(5))")
run("SELECT bin(-5), length(bin(-1)), length(bin((-1)::TINYINT))")
run("SELECT length(bin(9223372036854775807)), length(bin(18446744073709551615::UBIGINT))")
run("SELECT length(bin(170141183460469231731687303715884105727::HUGEINT)), "
    "length(bin(340282366920938463463374607431768211455::UHUGEINT))")
run("SELECT bin(5::TINYINT), bin(255::UTINYINT), bin('abc'), bin('12'), bin(5::VARINT)")
run("SELECT bin(-9223372036854775808::BIGINT)")
run("SELECT bin(1.5)")
run("SELECT bin('\\xAA'::BLOB)")

[('101', '0', '1', None, 'VARCHAR')]
[('1111111111111111111111111111111111111111111111111111111111111011', 64, 64)]
[(63, 64)]
[(127, 128)]
[('101', '11111111', '011000010110001001100011', '0011000100110010', '10000000000000000000000100000101')]
OSError: Type INT128 with value 9223372036854775808 can't be cast because the value is out of range for the destination type INT64
InternalError: Binder Error: No function matches the given name and argument types 'bin(DECIMAL(2,1))'. You might need to add explicit type casts.
InternalError: Binder Error: No function matches the given name and argument types 'bin(BLOB)'. You might need to add explicit type casts.

Examples

Each number and string in binary. to_binary gives the same result as bin, and -7 is written as 64 digits:

run(f"SELECT id, bin(n), to_binary(n), bin(s) FROM {TABLE} ORDER BY id LIMIT 10")

[(1, '11000000000', '11000000000', '0110100001101001'), (2, '101', '101', '01000001'), (3, '1111111111111111111111111111111111111111111111111111111111111001', '1111111111111111111111111111111111111111111111111111111111111001', '011000010110001001100011'), (4, None, None, None)]

The number of binary digits each non-negative number needs:

run(f"SELECT id, n, length(bin(n)) FROM {TABLE} WHERE n >= 0 ORDER BY id LIMIT 10")

[(1, 1536, 11), (2, 5, 3)]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). VAST DataBase can evaluate a comparison of a column with a constant, but not a function of a column, so bin() is always computed by the Query Engine:

• In the SELECT list, the scan reads the column and the Query Engine computes bin() for each row it returns.
• In WHERE, the scan sends the column for every row, and a FILTER step in the Query Engine computes bin() and keeps the rows that match. No rows are skipped in storage.

The plan of a WHERE condition on bin() shows the FILTER step above the scan:

plan(f"SELECT id FROM {TABLE} WHERE bin(n) = '101'")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #1            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│      (bin(n) = '101')     │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│             n             │
│             id            │
│                           │
└───────────────────────────┘

Make it faster

• Filter on the number, not its binary text. n = 5 gives the same rows as bin(n) = '101', and a comparison of the raw column with a constant is pushed down to VAST DataBase, so rows that don't match never leave storage. The plan has no FILTER step:

plan(f"SELECT id FROM {TABLE} WHERE n = 5")

┌───────────────────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│      Projections: id      │
│                           │
└───────────────────────────┘

• Compute it in SELECT. bin() in the SELECT list runs only on the rows that pass the filter.

Limitations and errors

• Integers and strings only. A DECIMAL, DOUBLE, FLOAT, BOOLEAN, DATE, INTERVAL, BLOB or list fails with No function matches the given name and argument types. Cast a number to BIGINT first.
• Negative numbers are written in 64 digits. A negative TINYINT, SMALLINT, INTEGER or BIGINT gives its 64-bit two's complement, whatever its own size. Use to_base for non-negative numbers padded to a fixed width; it doesn't take negative numbers.
• -9223372036854775808::BIGINT fails. In it, the cast applies before the minus sign, and 9223372036854775808 doesn't fit a BIGINT, so the query fails with Type INT128 with value 9223372036854775808 can't be cast because the value is out of range for the destination type INT64.
• Not for VARINT. bin(5::VARINT) returns '10000000000000000000000100000101', not '101'. A VARINT can't be cast to BIGINT either: the cast fails with Unimplemented type for cast (VARINT -> BIGINT). Keep numbers you write in binary in an integer type such as BIGINT or HUGEINT.

run("SELECT bin(1.5::DOUBLE)")
run("SELECT bin(CAST(1.5::DOUBLE AS BIGINT))")
run("SELECT CAST(5::VARINT AS BIGINT)")

InternalError: Binder Error: No function matches the given name and argument types 'bin(DOUBLE)'. You might need to add explicit type casts.
[('10',)]
OSError: Unimplemented type for cast (VARINT -> BIGINT)

Similar functions

run("SELECT bin(10), to_binary(10), to_base(10, 2), hex(10), bit_count(10)")
run("SELECT bin(-10), hex(-10)")
run("SELECT to_base(-10, 2)")
run("SELECT to_base(5, 2, 8), from_binary(bin('hi'))")

[('1010', '1010', '1010', 'A', 2)]
[('1111111111111111111111111111111111111111111111111111111111110110', 'FFFFFFFFFFFFFFF6')]
OSError: 'to_base' number must be greater than or equal to 0
[('00000101', b'hi')]

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• Numeric and math functions: all the numeric functions.
• to_base and hex.
• Where filters run and pushdowns.
• EXPLAIN: how to read a plan.