/ 
DBIO-ClickHouse-0.900004
/ DBIO::ClickHouse::Introspect::Util

NAME

DBIO::ClickHouse::Introspect::Util - ClickHouse type-string parser shared by the introspect submodules

VERSION

version 0.900004

DESCRIPTION

Leaf module holding the pure-data helpers shared by the system.* readers (DBIO::ClickHouse::Introspect::Tables, DBIO::ClickHouse::Introspect::Columns, DBIO::ClickHouse::Introspect::Indexes) and by DBIO::ClickHouse::Introspect itself. None of these subs talk to the database or hold state, so they are unit-testable without a server -- which matters here more than in the other drivers, because ClickHouse has no embedded mode and every live assertion costs a running server.

The centrepiece is "normalize_type", the inverse of the type map in DBIO::ClickHouse::DDL. Everything in this module exists so that the introspected model of a table the DDL layer created describes the same column metadata the DDL layer started from -- otherwise the diff layer sees phantom changes on every run.

Round-trip fidelity

normalize_type is written so that feeding its result back through DBIO::ClickHouse::DDL's _clickhouse_column_type reproduces the native type string it was given. Two cases cannot round-trip and are called out here rather than hidden:

  • String is many-to-one. The DDL layer collapses the whole varchar / char / text / blob / json family onto String, so introspection can only pick one representative. It picks text (unsized, which String is). A schema that declared varchar => 255 introspects back as text with no size. This is only a phantom change if a desired-state model is built directly from column_info; when the desired state is produced by deploying the generated DDL and introspecting it (the family's test-deploy-and-compare strategy) both sides normalize identically.

  • Explicitly numbered enums. DBIO::ClickHouse::DDL numbers enum members 1..N in declaration order. An Enum8 whose values are anything else (starting at 0, sparse, reordered) cannot be expressed as data_type => 'enum', so the whole type string is kept verbatim -- which does round-trip, via the DDL layer's pass-through for parameterized types -- and the member names are still exposed in extra->{list} for readers that want them.

Bare Date and bare DateTime used to be a third case: they are returned verbatim here, and DBIO::ClickHouse::DDL used to lowercase data_type before its type-map lookup and widen them to Date32 / DateTime64(3). It now treats an exact native spelling as a passthrough, so they round-trip like any other native name.

METHODS

normalize_type

my $n = DBIO::ClickHouse::Introspect::Util->normalize_type('LowCardinality(Nullable(String))');
# { native_type => 'LowCardinality(Nullable(String))',
#   data_type   => 'text',
#   is_nullable => 1,
#   extra       => { low_cardinality => 1 } }

Parses one native ClickHouse type string (as system.columns.type reports it) into the portable DBIO column metadata that produced it. Always returns a hashref with native_type, data_type, is_nullable and extra; size is present only when the type carries one.

The wrappers are peeled in the order DBIO::ClickHouse::DDL nests them: LowCardinality outside Nullable outside the base type. Array(T) is handled by the base-type parser, because Array is not a nullability wrapper -- it is the type.

Recognised shapes:

Int8 Int16 Int32 Int64        -> tinyint smallint integer bigint
UInt8 UInt16 UInt32 UInt64    -> same, extra => { unsigned => 1 }
Float32 / Float64             -> real / double precision
String                        -> text
FixedString(N)                -> char, size N, extra => { fixed_string => 1 }
Decimal(P, S)                 -> decimal, size [P, S]
Decimal32/64/128/256(S)       -> decimal, size [9|18|38|76, S]
Date32                        -> date
DateTime64(3)                 -> datetime,    size 3
DateTime64(3, 'UTC')          -> timestamptz, size 3, extra => { timezone }
DateTime64(N[, 'TZ'])         -> verbatim,    size N, extra => { timezone }
Bool                          -> boolean
UUID                          -> uuid
Enum8(...) / Enum16(...)      -> enum, extra => { list => [...] }
Array(T)                      -> "<T>[]"
anything else                 -> verbatim

Verbatim is not a failure mode: DBIO::ClickHouse::DDL passes an unmapped data_type through untouched, so IPv6, Int128, Map(String, UInt64) and friends survive a full round trip unchanged.

view_select

my $select = DBIO::ClickHouse::Introspect::Util->view_select($create_table_query);

The SELECT of a view, cut out of the CREATE [MATERIALIZED] VIEW ... AS SELECT ... statement system.tables.create_table_query holds (ClickHouse has no view_definition worth reading). Returns undef for anything that is not such a statement.

split_top_level

my @entries = DBIO::ClickHouse::Introspect::Util->split_top_level("toYYYYMM(ts), id");
# ('toYYYYMM(ts)', 'id')

Splits a comma-separated ClickHouse expression list on its top-level commas, ignoring commas nested in parentheses, brackets or single-quoted string literals. Entries are trimmed and empty ones dropped.

This is what turns system.tables' sorting_key / partition_key / primary_key strings and an Enum8 member list into usable lists -- a plain split /,/ would break Decimal(10, 2) and toYYYYMM(ts) apart.

unquote_string

my $value = DBIO::ClickHouse::Introspect::Util->unquote_string(q{'it\\'s'});
# "it's"

The inverse of "quote_string" in DBIO::ClickHouse::DDL::Emit: strips the surrounding single quotes and unescapes backslash-escaped backslashes and quotes. Returns undef when $string is not a single complete quoted literal -- 'a' || 'b' is an expression, not a string default, and must not be silently truncated to a.

SEE ALSO

AUTHOR

DBIO Authors

COPYRIGHT AND LICENSE

Copyright (C) 2026 DBIO Authors

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.