strlen

Applies to VAST Cluster 5.5.

Functions › Text basics › Length. Returns the number of bytes in a string's UTF-8 encoding. It has no aliases.

Syntax

strlen(s)

Returns: BIGINT.

Aliases: none.

Set up the examples

The examples connect with the ADBC driver, as in VAST Query Engine (ADBC), and work in a schema called docs_sql_fn_strlen. Set QE_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and VASTDB_BUCKET (SQL reference), and run the blocks in order in one Python session. Below each block is the output you should see.

This block connects, defines the run() and plan() helpers, and creates a table of five people. Two names have letters outside ASCII, and the last row has no values:

import os
import re

import adbc_driver_manager
import adbc_driver_manager.dbapi as dbapi
import adbc_driver_vastdb

BUCKET = os.environ["VASTDB_BUCKET"]
SCHEMA = f'"{BUCKET}/docs_sql_fn_strlen"'

conn = dbapi.connect(
    driver=adbc_driver_vastdb.get_driver_path(),
    db_kwargs={
        "vast.db.endpoint": os.environ["QE_ENDPOINT"],
        "vast.db.access_key": os.environ["AWS_ACCESS_KEY_ID"],
        "vast.db.secret_key": os.environ["AWS_SECRET_ACCESS_KEY"],
    },
    autocommit=True,
)

def server_message(e):
    """The server's message in a driver error, without the request ID or the echoed statement."""
    text = re.sub(r"\[req=0x[0-9a-f]+\]: ", "", str(e))
    found = (re.search(r'message: "(.*?)", request_id', text, re.S)
             or re.search(r"description: (.*)\)", text, re.S))
    message = found.group(1) if found else text.strip().splitlines()[-1]
    message = re.sub(r"^Plan query: '.*?' failed with: ", "", message, flags=re.S)
    message = re.split(r"\\n|\n", message)[0].replace('\\"', '"')
    return message.replace(BUCKET, "<bucket>")

def run(sql):
    """Run one statement. Print the rows it returns, or the error."""
    try:
        with conn.cursor() as cur:
            cur.execute(sql)
            rows = cur.fetchall() if cur.description else []
    except (adbc_driver_manager.Error, OSError) as e:
        print(f"{type(e).__name__}: {server_message(e)}")
    else:
        print(rows)

def plan(sql):
    """Print the plan without its ~N Rows lines: the table's row count, which can still be 0 just after an insert."""
    with conn.cursor() as cur:
        cur.execute("EXPLAIN " + sql)
        steps = cur.fetchall()[0][1]
    print("\n".join(line for line in steps.splitlines() if "Rows" not in line))

TABLE = f"{SCHEMA}.people"
run(f"CREATE SCHEMA IF NOT EXISTS {SCHEMA}")
run(f"CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, name VARCHAR, city VARCHAR)")
run(f"INSERT INTO {TABLE} VALUES "
    "(1, 'Ada Lovelace', 'London'), "
    "(2, 'Grace Hopper', 'New York'), "
    "(3, 'José Martí', 'Havana'), "
    "(4, 'Søren Kierkegaard', 'København'), "
    "(5, NULL, NULL)")

[]
[]
[(5,)]

What it does

strlen counts the bytes of the string's UTF-8 encoding. UTF-8 stores each Unicode code point in one to four bytes: one for ASCII letters, digits and punctuation, two for most accented Latin letters, three for symbols such as €, and four for emoji. So strlen equals length for an ASCII string and is larger as soon as the string holds anything else:

The same text can have different byte counts: é written as one code point takes two bytes, and e followed by a combining accent takes three. strlen counts what is stored, not what a reader sees.

strlen takes only VARCHAR. For the bytes of a BLOB, use octet_length; for the bits of a BIT value, use bit_length.

NULLs and edge cases

The same cases, as the cluster returns them. They use the table and helpers from Set up the examples:

run("SELECT strlen(NULL), strlen(''), strlen('é'), strlen('€'), strlen('🇫🇷'), strlen('e\u0301')")
run("SELECT strlen('101'::BIT)")
run("SELECT strlen('\\xAA'::BLOB)")
run("SELECT strlen(['a', 'b'])")
run("SELECT strlen(123)")

[(None, 0, 2, 3, 8, 3)]
InternalError: Binder Error: No function matches the given name and argument types 'strlen(BIT)'. You might need to add explicit type casts.
InternalError: Binder Error: No function matches the given name and argument types 'strlen(BLOB)'. You might need to add explicit type casts.
InternalError: Binder Error: No function matches the given name and argument types 'strlen(VARCHAR[])'. You might need to add explicit type casts.
InternalError: Binder Error: No function matches the given name and argument types 'strlen(INTEGER_LITERAL)'. You might need to add explicit type casts.

Examples

The characters and the bytes of each name and city. José Martí has 10 characters and 12 bytes, because é and í take two bytes each; Søren Kierkegaard and København each have one two-byte letter:

run(f"SELECT id, length(name), strlen(name), length(city), strlen(city) FROM {TABLE} ORDER BY id LIMIT 10")

[(1, 12, 12, 6, 6), (2, 12, 12, 8, 8), (3, 10, 12, 6, 6), (4, 17, 18, 9, 10), (5, None, None, None, None)]

Find the names that hold anything outside ASCII: for those, the byte count is larger than the character count:

run(f"SELECT id, name FROM {TABLE} WHERE strlen(name) > length(name) ORDER BY id LIMIT 10")

[(3, 'José Martí'), (4, 'Søren Kierkegaard')]

Check values against a byte limit, for example before you send them to a system that allows 16 bytes per field. Søren Kierkegaard is 17 characters but 18 bytes:

run(f"SELECT id, name, strlen(name) FROM {TABLE} WHERE strlen(name) > 16 ORDER BY id LIMIT 10")

[(4, 'Søren Kierkegaard', 18)]

How it runs in VAST

The Query Engine runs on the cluster's CNodes, and a query on a table starts with a scan in VAST DataBase (VAST_SCAN). VAST DataBase can evaluate a comparison of a column with a constant, but not a function of a column, so strlen() is always computed by the Query Engine:

• In the SELECT list, the scan reads the column and the Query Engine computes strlen() for each row it returns.
• In WHERE, the scan sends the column for every row, and a FILTER step in the Query Engine computes strlen() and keeps the rows that match. No rows are skipped in storage.

The plan of the byte-limit example shows the FILTER step above the scan:

plan(f"SELECT id FROM {TABLE} WHERE strlen(name) > 16")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #1            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│    (strlen(name) > 16)    │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│            name           │
│             id            │
│                           │
└───────────────────────────┘

Make it faster

• Test for empty strings with = '', not strlen(s) = 0. A comparison of the raw column with a constant is pushed down to VAST DataBase, so rows that don't match never leave storage. The plan for name = '' has no FILTER step:

plan(f"SELECT id FROM {TABLE} WHERE name = ''")

┌───────────────────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│      Projections: id      │
│                           │
└───────────────────────────┘

• Store the byte length if you filter on it often. Compute it when you write the rows and keep it in its own column: name_bytes > 16 is pushed down, strlen(name) > 16 isn't.
• Combine it with a pushed-down condition. In id > 2 AND strlen(name) > 16, id > 2 is still evaluated in VAST DataBase, and the FILTER step holds only the strlen() condition, so strlen() runs only on the rows that pass id > 2:

plan(f"SELECT id FROM {TABLE} WHERE id > 2 AND strlen(name) > 16")

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│             #0            │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│    (strlen(name) > 16)    │
│                           │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         VAST_SCAN         │
│    ────────────────────   │
│    Function: VAST_SCAN    │
│                           │
│        Projections:       │
│             id            │
│            name           │
│                           │
└───────────────────────────┘

Limitations and errors

• Strings only. A number, a date, a boolean, a list or a BIT value fails with No function matches the given name and argument types; the error lists strlen(VARCHAR) as the only form. For a BIT value, use bit_length.
• Not for BLOB. Use octet_length to count the bytes of a BLOB.
• octet_length doesn't take a VARCHAR. On a string constant it fails with Could not choose a best candidate function for the function call "octet_length(STRING_LITERAL)", because it's defined for BIT and BLOB only. Use strlen for the bytes of a string; octet_length(encode(s)), which converts the string to a BLOB first, gives the same count.

run("SELECT octet_length('abc')")
run("SELECT octet_length('\\xAA\\xBB'::BLOB)")

InternalError: Binder Error: Could not choose a best candidate function for the function call "octet_length(STRING_LITERAL)". In order to select one, please add explicit type casts.
[(2,)]

Similar functions

run("SELECT strlen('José'), length('José'), bit_length('José'), length_grapheme('José')")
run("SELECT strlen('e\u0301'), length('e\u0301'), bit_length('e\u0301'), length_grapheme('e\u0301')")

[(5, 4, 40, 4)]
[(3, 2, 24, 1)]

Clean up:

run(f"DROP TABLE IF EXISTS {TABLE}")
run(f"DROP SCHEMA IF EXISTS {SCHEMA}")
conn.close()

[]
[]

See also

• Text basics: all the text functions.
• length, bit_length and length_grapheme: the other ways to measure a string.
• octet_length: the bytes of a BLOB.
• Where filters run and pushdowns.
• EXPLAIN: how to read a plan.