<< (shift left)

Applies to VAST Cluster 5.5.

Functions › Operators and predicates › Bitwise operators. a << n shifts the bits of a left by n places, which multiplies a by 2 to the power n: 1 << 4 is 16. A result that doesn't fit the type, a negative a and a negative n are errors.

Syntax

a << n

Returns: the integer type of the arguments: 1 << 4 is an INTEGER. For a BIT value, a BIT.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_operators_shift_left. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of four values n, each with a number of places k to shift it by. The last row has no values:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_operators_shift_left"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.shifts"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, n INTEGER, k INTEGER)")
run(f"INSERT INTO {TABLE} VALUES (1, 1, 4), (2, 3, 2), (3, 5, 0), (4, NULL, NULL)")

[]
[]
[(4,)]

What it does

a << n moves every bit of a up by n places and fills the places it leaves with zeros, so the result is a times 2 to the power n: 3 << 4 is 48, the same as 3 * 16.

The result has the type of the arguments, and it must fit that type. The shift doesn't wrap around or drop bits:

• A result too large for the type fails with Overflow in left shift. For a signed type, the result can't reach the sign bit: 1 << 30 is fine in an INTEGER, but 1 << 31 fails. An unsigned type can use its top bit: 1::UTINYINT << 7 is 128.
• A shift by the whole width or more fails. For a signed type the error is Left-shift value 64 is out of range (for a BIGINT); for an unsigned type it's Overflow in left shift. 0 << 100 is 0.
• A negative a or a negative n fails, with Cannot left-shift negative number -1 or Cannot left-shift by negative number -1.

To shift further, cast a to a wider type first: 1::BIGINT << 62 works.

+ binds more tightly than <<: 1 << 2 + 1 is 1 << 3, which is 8.

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT 1 << 4, 1 << 30, 1 << 0, 0 << 100, 1 << 2 + 1, NULL << 1, 1 << NULL")
run("SELECT 1 << 31")
run("SELECT 1::BIGINT << 62")
run("SELECT 1::BIGINT << 63")
run("SELECT 1::BIGINT << 64")
run("SELECT 1::UTINYINT << 7, 1::UBIGINT << 63")
run("SELECT 1::UTINYINT << 8")
run("SELECT 3::TINYINT << 6")
run("SELECT 1::HUGEINT << 126")
run("SELECT -1 << 1")
run("SELECT 1 << -1")
run("SELECT 1.5 << 1")
run(f"SELECT '12' << k FROM {TABLE} LIMIT 1")

[(16, 1073741824, 1, 0, 8, None, None)]
OSError: Overflow in left shift (1 << 31)
[(4611686018427387904,)]
OSError: Overflow in left shift (1 << 63)
OSError: Left-shift value 64 is out of range
[(128, 9223372036854775808)]
OSError: Overflow in left shift (1 << 8)
OSError: Overflow in left shift (3 << 6)
[(Decimal('85070591730234615865843651857942052864'),)]
OSError: Cannot left-shift negative number -1
OSError: Cannot left-shift by negative number -1
InternalError: Binder Error: No function matches the given name and argument types '<<(DECIMAL(2,1), INTEGER_LITERAL)'. You might need to add explicit type casts.
InternalError: Binder Error: Could not choose a best candidate function for the function call "<<(STRING_LITERAL, INTEGER)". In order to select one, please add explicit type casts.

Examples

Shift each value by its own number of places:

run(f"SELECT id, n, k, n << k FROM {TABLE} ORDER BY id LIMIT 10")

[(1, 1, 4, 16), (2, 3, 2, 12), (3, 5, 0, 5), (4, None, None, None)]

Build a bit mask with one bit set, and keep the rows whose shifted value is larger than 10:

run(f"SELECT id, 1 << k FROM {TABLE} ORDER BY id LIMIT 10")
run(f"SELECT id FROM {TABLE} WHERE (n << k) > 10 ORDER BY id LIMIT 10")

[(1, 16), (2, 4), (3, 1), (4, None)]
[(1,), (2,)]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). VAST DataBase can evaluate a comparison of a column with a constant, but not an operation on a column, so << on a column is always computed by the Query Engine:

• In the SELECT list, the scan reads the columns and the Query Engine computes << for each row it returns.
• In WHERE, the scan sends the columns for every row, and a FILTER step in the Query Engine computes << and keeps the rows that match. No rows are skipped in storage.

An error on one row cancels the whole query, in SELECT and in WHERE: one negative value in the column is enough. The plan of the WHERE example shows the FILTER step above the scan:

plan(f"SELECT id FROM {TABLE} WHERE (n << k) > 10")
run("SELECT n << k FROM (VALUES (1, 1), (-18, 2)) v(n, k)")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #2            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│      ((n << k) > 10)      │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│             n             │
│             k             │
│             id            │
│                           │
└───────────────────────────┘
OSError: Cannot left-shift negative number -18

Make it faster

• Shift the constant, not the column. A shift of constants is computed before the scan, so n = 1 << 10 is a comparison of the raw column with a constant, which is pushed down to VAST DataBase. The plan has no FILTER step:

plan(f"SELECT id FROM {TABLE} WHERE n = 1 << 10")

┌───────────────────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│      Projections: id      │
│                           │
└───────────────────────────┘

Limitations and errors

• Overflow is an error. A result that doesn't fit the type fails with Overflow in left shift, and the whole query stops. Cast a to a wider type, such as BIGINT or HUGEINT, before you shift.
• No negative values. A negative a fails with Cannot left-shift negative number, and a negative n with Cannot left-shift by negative number. To multiply a negative number by a power of 2, use *: -3 * 16.
• Integers and BIT only. A DECIMAL, DOUBLE, FLOAT, BOOLEAN, DATE, INTERVAL, BLOB or list fails with No function matches the given name and argument types. A string literal fails with Could not choose a best candidate function, because it could be a BIT or an integer; cast it.
• No shiftleft function. shiftleft(1, 4) fails with Scalar Function with name shiftleft does not exist!. Use <<.

run("SELECT -3 * 16, CAST('12' AS INTEGER) << 1")
run("SELECT shiftleft(1, 4)")

[(-48, 24)]
InternalError: Catalog Error: Scalar Function with name shiftleft does not exist!

Similar functions

run("SELECT 3 << 4, 3 * 16, 3 * pow(2, 4), 3 >> 4")

[(48, 48, 48.0, 0)]

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• Operators and predicates: all the operators.
• >> (shift right), & (bitwise AND).
• Where filters run and pushdowns.
• EXPLAIN: how to read a plan.