length

Applies to VAST Cluster 5.5.

Functions › Text basics › Length. Returns the number of characters in a string, the number of bits in a BIT value, or the number of elements in a list. len is the same function.

Syntax

length(s)
len(s)

Returns: BIGINT.

Aliases: len.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_length. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of four people. The last row has no values:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_length"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.people"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, name VARCHAR, city VARCHAR, tags VARCHAR[])")
run(f"INSERT INTO {TABLE} VALUES "
    "(1, 'Ada Lovelace', 'London', ['math', 'computing']), "
    "(2, 'Grace Hopper', 'New York', ['computing']), "
    "(3, 'José Martí', 'Havana', []), "
    "(4, NULL, NULL, NULL)")

[]
[]
[(4,)]

What it does

length counts the units a value is made of:

A code point isn't always what a reader sees as one character. A letter followed by a combining accent (e and U+0301, displayed as é) is two code points, and so is a flag emoji. To count what a reader sees, use length_grapheme. To count bytes, use strlen.

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT length(NULL), length(''), length('e\u0301'), length('🇫🇷'), length([]), len('abc')")
run("SELECT length(123)")
run("SELECT length('\\xAA'::BLOB), octet_length('\\xAA'::BLOB)")

[(None, 0, 2, 2, 0, 3)]
InternalError: Binder Error: No function matches the given name and argument types 'length(INTEGER_LITERAL)'. You might need to add explicit type casts.
InternalError: Binder Error: No function matches the given name and argument types 'length(BLOB)'. You might need to add explicit type casts.

Examples

The length of each name and city, and the number of tags. José Martí has 10 characters; strlen counts 12 bytes, because é and í take two bytes each in UTF-8:

run(f"SELECT id, length(name), len(city), length(tags), strlen(name) FROM {TABLE} ORDER BY id LIMIT 10")

[(1, 12, 6, 2, 12), (2, 12, 8, 1, 12), (3, 10, 6, 0, 12), (4, None, None, None, None)]

Keep only the rows with a long name:

run(f"SELECT id, name FROM {TABLE} WHERE length(name) > 10 ORDER BY id LIMIT 10")

[(1, 'Ada Lovelace'), (2, 'Grace Hopper')]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). VAST DataBase can evaluate a comparison of a column with a constant, but not a function of a column, so length() is always computed by the Query Engine:

• In the SELECT list, the scan reads the column and the Query Engine computes length() for each row it returns.
• In WHERE, the scan sends the column for every row, and a FILTER step in the Query Engine computes length() and keeps the rows that match. No rows are skipped in storage.

The plan of the WHERE example shows the FILTER step above the scan:

plan(f"SELECT id FROM {TABLE} WHERE length(name) > 10")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #1            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│    (length(name) > 10)    │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│            name           │
│             id            │
│                           │
└───────────────────────────┘

Make it faster

• Test for empty strings with = '', not length(s) = 0. A comparison of the raw column with a constant is pushed down to VAST DataBase, so rows that don't match never leave storage. The plan for name = '' has no FILTER step:

plan(f"SELECT id FROM {TABLE} WHERE name = ''")

┌───────────────────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│      Projections: id      │
│                           │
└───────────────────────────┘

• Store the length if you filter on it often. Compute it when you write the rows and keep it in its own column: name_length > 10 is pushed down, length(name) > 10 isn't.
• Combine it with a pushed-down condition. In id > 100 AND length(name) > 10, id > 100 is still evaluated in VAST DataBase, so length() runs only on the rows that pass it.

Limitations and errors

• Strings, bits and lists only. A number, a date or a boolean fails with No function matches the given name and argument types. The error lists the types length accepts: length(VARCHAR), length(BIT) and length(ANY[]).
• Not for BLOB. Use octet_length to count the bytes of a BLOB. octet_length doesn't take a VARCHAR; for the bytes of a string, use strlen.
• Code points, not characters. See What it does.
• Other SQL dialects' names aren't available. char_length and character_length fail with Scalar Function with name ... does not exist!. Use length.

run("SELECT char_length('abc')")

InternalError: Catalog Error: Scalar Function with name char_length does not exist!

Similar functions

run("SELECT length('José'), strlen('José'), bit_length('José'), length_grapheme('José')")
run("SELECT length('e\u0301'), strlen('e\u0301'), bit_length('e\u0301'), length_grapheme('e\u0301')")

[(4, 5, 40, 4)]
[(2, 3, 24, 1)]

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• Text basics: all the text functions.
• Where filters run and pushdowns.
• EXPLAIN: how to read a plan.