Release v0.036

Tags:

In this release:

Added

  • Added parse_time function to yq. This function parses a date/time string into a UNIX epoch.
  • Added epoch timestamps as an allowed format for yts time series timestamps.
  • Add default timestamps of "now" in yts. This makes all time series databases the same and makes it easier to add metrics.
  • Added 'LINUX.PROC.LOADAVG' and 'LINUX.PROC.UPTIME' patterns to ygrok to parse the /proc/loadavg and /proc/uptime files on Linux systems.

Fixed

  • Reduced version requirement for List::Util. This continues our quest to be installable on a core Perl 5.10 without a compiler.
  • Removed unused prereq (Text::Trim).

Removed

  • Removed the ymask command and associated prereqs. This command just wasn't very useful, and depended on a module that requires a compiler.

More information about ETL-Yertl v0.036 on MetaCPAN

Release v0.035

Tags:

In this release:

More information about ETL-Yertl v0.035 on MetaCPAN

Added

  • Added Graphite time series adapter. Now yts can read/write time series from the Graphite system and related subsystems. The write function is compatible with the "line" protocol, so any databases that understand Graphite's line protocol can be written to.

  • Added --start and --end options to yts command to constrain the time series returned to a specific date/time range.

Fixed

  • Removed 'Z' time zone indicator from yts input/output and time series adapters. Time zone does not enter in to any of the things we're doing with time series. If users need to do things with time zones, they'll have to do it themselves (poor souls).

  • Fixed a race condition that could cause yts to read the data and not write the data passed-in on STDIN. This requires a new dependency, IO::Interactive.

  • Removed dependencies on some modules to reduce the memory footprint and improve startup times:

    • DateTime
    • Moo
    • Type::Tiny

  • Removed some unused modules that are not going to be developed anymore.

    • Parse::RecDescent versions of the yq language parser. Perl 5.10 regex grammars work just fine.

  • Fixed YAML.pm (YAML::Old) causing tests to fail. This has been removed (for now) as a supported YAML module. The default supported YAML module is now YAML::Tiny.

  • Fixed the yts time series format. Now there is only the "metric" to identify the time series, which for some time series databases may contain abstractions like "database" and "column".

  • Fixed nulls appearing in Graphite time series.

Release v0.032

Tags:

In this release, we've added a command to read/write time series data: yts. It only works with InfluxDB for now, but writing new interfaces to time series databases is easy!

Along with the new ysql --count option, it's easy to create metrics from data in your database:

# Count the number of test reports
ysql cpantesters --count test_reports \
    | yts influxdb://localhost/telegraf cpantesters report_count

# Count the number of processed reports
ysql cpantesters --count cpanstats \
    | yts influxdb://localhost/telegraf cpantesters processed_count

# Build metrics for Minion job worker
ysql cpantesters --count minion_jobs \
    | yts influxdb://localhost/telegraf minion total_jobs
ysql cpantesters --count minion_jobs --where 'status="inactive"' \
    | yts influxdb://localhost/telegraf minion inactive_jobs
ysql cpantesters --count minion_jobs --where 'status="finished"' \
    | yts influxdb://localhost/telegraf minion finished_jobs

More information about ETL-Yertl v0.032 on MetaCPAN

Release v0.029

Tags:

In this release:

Added

  • yq's filter language now allows $. to refer to the initial document even if the current document (.) is something else. This way one can traverse the document using | filters and still get to the original.
  • yq's filter language now allows assigning values to filters, so you can modify documents in-place.
  • yq's filter language now has an each() function to iterate over hashes/arrays
  • yq's filter language now allows function calls on either side of binary operators (assignment operator and comparison operators). This means we can filter based on the output of a function (like find all arrays with more than 5 elements) or assign the output of a function to a place in the document (like replace an array with its length).

Fixed

  • We now give better error messages when accidentally trying to insert hashrefs or arrayrefs into a column using the ysql --insert helper. SQL::Abstract does not allow this and instead gives an untrappable warning (instead of throwing an exception, which is potentially earmarked for SQL::Abstract v2)

More information on ETL::Yertl 0.029 on MetaCPAN

Feeds