deon
DeObject Notation Format
deon is a notation format for structured data.
deon is intended to be:
- light on syntax — friendly for human read/write, should feel more like note-taking than data entry;
- moderately fast — with a general use case for configuration-like files, loaded once at build/runtime;
- programming-lite — although not a programming language, the in-file imports and the linking (in-file variables) give
deon a programmatic feel.
The deon filename extension is .deon, and the media type is application/deon.
Why deobject? More of a play-on-words, although a case can be made considering the linking feature and the possible 'assembling' of the root, as if the object has been de-structured.
Contents
Example
The following .deon file
// deon
{
entities [
{
id 01
name One
active true
}
{
id 02
name Two
active false
}
]
#time
}
time 1598439736
will produce the following data
// JavaScript/TypeScript
const data = {
entities: [
{
id: '01',
name: 'One',
active: 'true',
},
{
id: '02',
name: 'Two',
active: 'false',
},
],
time: '1598439736',
};
// Rust
let data = deon!({
entities [
{
id 01
name One
active true
}
{
id 02
name Two
active false
}
]
time: 1598439736
});
# Python
data = {
"entities": [
{
"id": "01",
"name": "One",
"active": "true",
},
{
"id": "02",
"name": "Two",
"active": "false",
},
],
"time": "1598439736",
}
Comparisons
Consider the following commonly-used formats with an example file from performer:
# an .yaml file
---
stages:
- name: 'Setup NPM Private Access'
directory: '/path/to/package'
imagene: 'ubuntu'
command:
- '/bin/bash'
- './configurations/.npmrc.sh'
secretsEnvironment:
- 'NPM_TOKEN'
- name: 'Generate the Imagene'
directory: '/path/to/package'
imagene: 'docker'
command: [
'build',
'-f',
'./configurations/docker.development.dockerfile',
'-t',
'hypod.cloud/package-name:$SHORT_SHA',
'.'
]
- name: 'Push Imagene to Registry'
directory: '/path/to/package'
imagene: 'docker'
command: [
'push',
'hypod.cloud/package-name:$SHORT_SHA'
]
timeout: 720
// a .json file
{
"stages": [
{
"name": "Setup NPM Private Access",
"directory": "/path/to/package",
"imagene": "ubuntu",
"command": [
"/bin/bash",
"./configurations/.npmrc.sh"
],
"secretsEnvironment": [
"NPM_TOKEN"
]
},
{
"name": "Generate the Imagene",
"directory": "/path/to/package",
"imagene": "docker",
"command": [
"build",
"-f",
"./configurations/docker.development.dockerfile",
"-t",
"hypod.cloud/package-name:$SHORT_SHA",
"."
]
},
{
"name": "Push Imagene to Registry",
"directory": "/path/to/package",
"imagene": "docker",
"command": [
"push",
"hypod.cloud/package-name:$SHORT_SHA"
]
}
],
"timeout": 720
}
Consider the .deon version:
// a .deon file
{
stages [
{
name Setup NPM Private Access
directory /path/to/package
imagene ubuntu
command [
/bin/bash
./configurations/.npmrc.sh
]
secretsEnvironment [
NPM_TOKEN
]
}
{
name Generate the Imagene
directory /path/to/package
imagene docker
command [
build
-f
./configurations/docker.development.dockerfile
-t
hypod.cloud/package-name:$SHORT_SHA
.
]
}
{
name Push Imagene to Registry
directory /path/to/package
imagene docker
command [
push
hypod.cloud/package-name:$SHORT_SHA
]
}
]
timeout 720
}
or with nested internal linking
// a .deon file
// the root
{
stages [
#stage1
#stage2
#stage3
]
timeout 720
}
// the leaflinks
stage1 {
name Setup NPM Private Access
#directory
imagene ubuntu
command #commands.stage1
#secretsEnvironment
}
stage2 {
name Generate the Imagene
#directory
imagene docker
command #commands.stage2
}
stage3 {
name Push Imagene to Registry
#directory
imagene docker
command #commands.stage3
}
directory /path/to/package
commands {
stage1 [
/bin/bash
./configurations/.npmrc.sh
]
stage2 [
build
-f
./configurations/docker.development.dockerfile
-t
#imageneName
.
]
stage3 [
push
#imageneName
]
}
secretsEnvironment [
NPM_TOKEN
]
imageneName hypod.cloud/package-name:$SHORT_SHA
Implementations
deon is implemented for:
and will be implemented for:
See specifics for implementation details.
Installs
deon can be installed locally with the appropriate package manager for each implementation language, or can be installed globally as a Command-Line Interface tool.
Using the NodeJS runtime, run the command
npm install -g @plurid/deon
or download the appropriate binary
CLI
Usage: deon [options] [command] <file>
read a ".deon" file and output the parsed result
Options:
-v, --version output the version number
-o, --output <type> output type: deon, json (default: "deon")
-t, --typed typed output (default: false)
-h, --help display help for command
Commands:
convert <source> [destination] convert a ".json" file to ".deon"
environment [options] <source> <command...> loads environment variables from a ".deon" file and spawns a new command
Service
The deon data parsing can be explored on deon.plurid.com and deon.plurid.com/parse can be used for programmatic POST requests.
curl \
-X POST \
-H 'Content-Type: application/deon' \
-d '{ key value }' \
https://deon.plurid.com/parse
The response is deon by default, but can be specified through the kind query parameter (json, yaml, toml, or xml).
curl \
-X POST \
-H 'Content-Type: application/deon' \
-d '{ key value }' \
https://deon.plurid.com/parse?kind=yaml
Conversely, json, yaml, toml, or xml data can be converted to deon through a deon.plurid.com/convert POST request
curl \
-X POST \
-H 'Content-Type: application/json' \
-d '{ "key": "value" }' \
https://deon.plurid.com/convert
The deon.plurid.com service can also host .deon files to be easily imported or injected into other files from a fixed URL allowing control over the private/public status, the file's revision, or access-aware content (the content of the file changes based on who/what token requests it).
General
A deon is comprised of a required root and none or more, optional leaflinks.
In deon every endleaf value is a string. It is up to the consumer to handle the required type conversions based on the problem domain interface. An advanced use case couples deon with datasign to handle type conversions.
deon supports two types of value groupings, the map and the list.
The root can be map or list-like.
The leaflinks can be strings, maps, or lists.
The maps and the lists can have strings, lists, and maps as values,
The order of the root or of any of the leaflinks is not important.
The per map key names and the leaflinks names are expected to be unique.
When parsed or imported, a .deon file will allow access only to the root. The leaflinks are private as data-details at the file level. By convention, a __leaflinks__ key can be manually added to the root to allow access to the leaflinks if absolutely needed.
Values
An endleaf value, simply called value, is a string of characters, with or without spaces:
{
key simpleValue
}
{
key value with spaces
}
A value can be surrounded by singlequotes ' in order to support special characters, such as trailing spaces
{
key 'value with 4 trailing spaces '
}
Multi-line values are surrounded by backticks `. The multi-line string is stripped of any whitespace or new lines before the first non-space character and after the last non-space character.
{
key `
a
multi
line
string
value
`
}
or linked
{
#key
}
key `
a
multi
line
string
value
`
Maps
mapName {
mapKey mapValue
}
A map is comprised of key-value pairs. The deon root is the single base-level map without a mapName.
A mapKey is an A-Za-z0-9_- string of characters. To support special characters (such as space), the mapKey must be surrounded by single quotes, such as
mapName {
'map Key' mapValue
}
A mapValue starts after the space of the mapKey and continues until the end of the line or until a comma.
mapName {
mapKey1 map Value 1
mapKey2 mapValue2
}
or
mapName {
mapKey1 map Value 1, mapKey2 mapValue2
}
A mapValue can be a string, a list, or a map.
A mapValue can be an empty string:
mapName {
mapKey1
mapKey2 mapValue2
}
or
mapName {
mapKey1 '', mapKey2 mapValue2
}
Lists
listName [
list Value 1
listValue2
]
A list is comprised of a listName and the list items. The deon root is the single base-level list without a listName.
A list item value starts at the first non-space character after the left square bracket [, or after the previous list item, and ends at the end of the line or at the comma.
Such as
listName [
list Value 1, listValue2
]
or
listName [list Value 1, listValue2]
Each list item can be a string, a list, or a map.
A list item can be an empty string:
listName [
''
listValue2
]
or
listName [
'', listValue2
]
Structures
A structure is used to specify structured data
{
aStructure <
// structure signature
id
value
> [
// first data entry
one
two
// second data entry
three
four
// third data entry
five
six
]
}
Single-line comments use the doubleslash //.
// comment outside root
{
// comment inside root
key value // comment in-line
}
Multi-line comments use the slashstar /* to start, and the starslash to end */.
/*
multi
line
comment outside root
*/
{
/*
multi
line
comment inside root
*/
key value
}
Linking
General
A leaflink is designated using the hash sign #.
The .deon file
{
key value
}
can be linked thus
{
key #arbitraryName
}
arbitraryName value
or with shortened linking
{
#key
}
key value
To support linking with special characters in name, the leaflink must be surrounded by singlequotes '.
{
#'key with spaces'
}
'key with spaces' value
Dot-access
A leaflink can be dot-accessed:
{
entities [
{
name #entity1.name
}
]
}
#entity1 {
name The Entity
}
or dot-accessed with shortened link
{
entities [
{
#entity1.name
}
]
}
#entity1 {
name The Entity
}
in which case, the key will be the last key of the dot-access string.
Name-access
A leaflink can be name-accessed:
{
entities [
{
name #entity1[name]
}
]
}
#entity1 {
name The Entity
}
or from a list:
{
entities [
{
name #names[0]
}
]
}
#names [
one
two
three
]
The list has a zero-based indexation.
Spreading
A leaflink can be spreaded by tripledots ...:
{
entities [
{
...#entity1
}
]
}
#entity1 {
name The Entity
timestamp 1598425060
}
Spreading overwrites the previously defined keys, if any, with the same name as the keys in the spreaded map.
A map can be spreaded only in another map. A list can be spreaded only in another list. A string can be spreaded in a map and will result in a map where each key equals the index of the character of the string, or can be spreaded into a list and will result in a list where each list item is a character of the string.
{
entity {
...#spread
}
}
spread abc
wil result in entity having the value:
entity {
0 a
1 b
2 c
}
whereas
{
entity [
...#spread
]
}
spread abc
wil result in entity having the value:
entity [
a
b
c
]
Environment variables
A leaflink can represent an environment variable using the #$ syntax. The environment variable will be injected at parse-time:
{
one #$SOME_ENV_VARIABLE
}
two #$ANOTHER_ENV_VARIABLE
Importing
A .deon file can import another .deon file using the following syntax
import <name> from <path>
Where the name is an arbitrary string, and the path is the path of the targeted .deon file.
The path does not need to have the .deon filename extension specified.
The path can also point to a .json file, and deon will parse it appropriately.
The import imports the root from the targeted .deon file in order to be used as a regular, in-file locally-defined leaflink.
The import statement order in file is not important, although, by convention, they sit at the top of file. Imports will be resolved primarily, before any other action. The import name must be unique among all the other imports and among the in-file locally-defined leaflinks, given that there is no discernible conceptual difference between them.
// file-1.deon
{
name The Name
}
// file-2.deon
import file1 from ./file-1
{
name #file1.name
}
The paths of the imported files can be relative filesystem paths, and they will be automatically searched and imported if found, or absolute filesystem paths, if all the used absolute paths are passed to the parser at parse-time.
// file-2.deon
import file1 from absolute/path/file-1
{
name #file1.name
}
and parsed giving the absolute paths, specific to a file or general using the /* glob-like matcher:
// TypeScript example
import Deon from '@plurid/deon';
const deonFilePath = '/absolute/path/to/folder/file-1.deon';
const deonFilesPath = '/absolute/path/to/folder/';
const loadData = async () => {
const absolutePaths = {
// specific file
'absolute/path/file-1': deonFilePath,
// or file lookup at runtime
'absolute/path/*': deonFilesPath,
};
const deon = new Deon();
const data = await deon.parseFile(
'/path/to/file-1.deon',
{
absolutePaths,
},
);
return data;
}
const main = async () => {
const data = await loadData();
// use data
console.log(data);
// { name: 'The Name' };
}
main();
A path can also be an URL such as
// file-url.deon
import urlFile from https://example.com/url-file.deon
{
#urlFile.key
}
In order to request URL files from protected routes, an authorization map of authorization tokens can be passed at parse-time with all the domains required by the imports
authorization {
example.com token
}
with the token being automatically passed into the Authorization: Bearer <token> header of the adequate domain at request-time.
// TypeScript example
import Deon from '@plurid/deon';
const loadData = async () => {
const authorization = [
'example.com': 'token', // provide token securely using environment variables
];
const deon = new Deon();
const data = await deon.parseFile(
'/path/to/file-url.deon',
{
authorization,
},
);
return data;
}
const main = async () => {
const data = await loadData();
// use data
console.log(data);
// { key: 'data from url file' };
}
main();
The token can be passed at import-time in the .deon file:
import urlFile from https://example.com/url-file.deon with secret-token
{
#urlFile.key
}
In order not to leak secrets, environment variables should be used:
import urlFile from https://example.com/url-file.deon with #$SECRET_TOKEN
{
#urlFile.key
}
Injecting
The import statement will always try to parse the filetext into structured data.
In order to get only the filetext, the keyword inject can be used:
inject leaflinkName from /path/to/file.any
{
key #leaflinkName
}
The arbitrarily-named inject entity can be used as a regular leaflink containing a string.
Similar to the import statement, the inject can target an URL and pass an optional authentication token.
inject file from https://example.com/file
inject secretFile from https://example.com/secret-file with secret-token
{
key1 #file
key2 #secretFile
}
In order to keep the .deon file secret-free, the secrets can be injected from a file outside or ignored by the versioning system.
inject secret from file-with-secret.text
inject secretFile from https://example.com/secret-file with #secret
{
key #secretFile
}
Interpolation
A string value can be interpolated in another string using the #{} syntax.
{
key value1 #{value2Key} value3
list [
value1 #{value2Key}
value3
]
}
value2Key value2-text
which will produce the result
{
key value1 value2-text text3
list [
value1 value2-text
value3
]
}
A deon entity, map, list, string, can be "called" to provision the interpolation values dynamically, on a use-case basis.
aKey {
subKey value1 #{value2} value3
}
{
key1 #aKey(
value2 value2-text
)
key2 #aKey(
value2 value2-different-text
)
}
Stringifying
A stringify method is implemented in order to convert an in-memory data representation to string. A partial options object can be passed.
interface DeonStringifyOptions {
readable: boolean;
indentation: number;
leaflinks: boolean;
leaflinkLevel: number;
leaflinkShortening: boolean;
generatedHeader: boolean;
generatedComments: boolean;
}
Parsing
The parse method can receive the following partial options:
interface DeonParseOptions {
absolutePaths: Record<string, string>,
authorization: Record<string, string>,
datasignFiles: string[];
datasignMap: Record<string, string>;
}
Literals
To handle deon data inside the implementation language, a language-specific literal can be used.
Javascript/Typescript
import {
deon,
} from '@plurid/deon';
const main = async () => {
const data = await deon`
// handles full-fledged deon data
// with imports, injects, leaflinks, etc.
{
key value
}
`;
// { key: 'value' }
console.log(data);
}
main();
Rust
use deon::{deon}
fn main() {
let data = deon!(
// handles full-fledged deon data
// with imports, injects, leaflinks, etc.
{
key value
}
);
// { key: 'value' }
println!("{}", data.to_string());
}
Advanced Usage
Datasign Type Conversion
When handling the parsing of .deon data, a .datasign file can be passed to handle the type conversions.
For example, given a JavaScript/TypeScript use case:
// ./Entity.datasign
data Entity {
name: string;
age: number;
}
// ./entity.deon
{
entities: [
{
name Entity One
age 1
}
{
name Entity Two
age 1.3
}
]
}
// ./index.ts
import Deon from '@plurid/deon';
const deonFile = './entity.deon';
const datasignFile = './Entity.datasign';
const data = Deon.parse(
deonFile,
{
// pass an array of all the .datasign files to be considered for type handling
datasignFiles: [
datasignFile,
],
// pass an object of the mappings between the fields from the .deon file
// and the expected types from the .datasign file
datasignMap: {
entities: 'Entity[]',
},
},
);
In Use
deon is used in:
Usages
deon can be plugged in into:
Idiomaticity
It appears idiomatic to have three sections in a .deon file, ordered as:
- imports;
- root;
- leaflinks.
The imports feel well-written when written in one line.
The root feels well-written when it has only one level of indentation, and every leaf is a leaflink (for maps or lists) or a string.
For example, the following joiner file:
import otherPackages from ../../path/to/file
{
#packages
#package
#commit
}
packages [
one
two
...#otherPackages
]
package {
manager yarn
publisher npm
}
commit {
engine git
combine true
root packages
fullFolder true
divider ' > '
message setup: package
}
Specifics
JavaScript / TypeScript
Parsing
The JavaScript / TypeScript can be used in the NodeJS runtime through the Deon object, or the deon template literal.
import Deon, {
deon,
} from '@plurid/deon';
The parsing of deon data can be achieving asynchronously or synchronously
import Deon, {
deon,
deonSynchronous,
} from '@plurid/deon';
const main = async () => {
const deonData = `
{
key value
}
`;
const deonObject = new Deon();
const parsedObjectAsynchronously = await deonObject.parse(deonData);
const parsedObjectSynchronously = deonObject.parseSynchronous(deonData);
const parsedTemplateAsynchronously = await deon`
{
key value
}
`;
const parsedTemplateSynchronously = deonSynchronous`
{
key value
}
`;
}
Synchronous parsing is to be used when the deon data does not rely on import or inject features, naturally asynchronous operations. However, when the parsing operation is to be used in a blockable environment (such as the CLI), synchronous parsing can be used for deon data with imports and injects just as well as asynchronous parsing.
deon data can also be parsed in the browser, or other sandboxed environments, using the DeonPure object, or the deonPure template literal.
import {
DeonPure,
deonPure,
deonPureSynchronous,
} from '@plurid/deon';
const main = async () => {
const deonData = `
{
key value
}
`;
const deonObject = new DeonPure();
const parsedObjectAsynchronously = await deonObject.parse(deonData);
const parsedObjectSynchronously = deonObject.parseSynchronous(deonData);
const parsedTemplateAsynchronously = await deonPure`
{
key value
}
`;
const parsedTemplateSynchronously = deonPureSynchronous`
{
key value
}
`;
}
The deon Pure implementation does not have access to the file system for import and inject features.
Typing
In order to handle the typing of the deon parsed data the typer can be used, which handles the typing in the standard JavaScript/TypeScript fashion, or the customTyper can be used, which requires an aditional, custom typing function.
import Deon, {
customTyper,
typer,
} from '@plurid/deon';
const main = async () => {
const deonData = `
{
keyBoolean true
keyNumber 1
}
`;
const deonObject = new Deon();
// keyBoolean and keyNumber are typeof 'string'
const parsedData = await deonObject.parse(deonData);
// keyBoolean is typeof 'boolean' and keyNumber is typeof 'number'
const defaultTypedData = typer(parsedData);
// keyBoolean is typeof 'boolean' and keyNumber is typeof 'string'
const customTypedData = customTyper(
parsedData,
(value) => {
if (value === 'true') {
return true;
}
return value;
},
);
}
Environment Loading
deon can be used to load environment variables at runtime from a .deon file.
Run the function call as soon as possible in the program.
// env-file.deon
{
ONE one
}
// index.ts
import Deon from '@plurid/deon';
const loadEnvironment = async () => {
const deon = new Deon();
// optional
const options = {
overwrite: false,
};
await deon.loadEnvironment(
'/path/to/env-file.deon'
options,
);
}
const main = async () => {
await loadEnvironment();
console.log(process.env.ONE) // one
}
main();
The .deon file that will be used for environment variables can use all the features of deon, however the root must be comprised only of strings, or list of strings, other values will be ignored.
Rust
In order to parse deon data the Deon implementation or the deon! macro can be used.
use deon::{
Deon,
deon!,
}
fn main() {
let deon_data = "
{
key value
}
";
let deon_object = Deon::new();
let deon_object_parsed = Deon::parse(deon_data);
let deon_macro_parsed = deon!(
{
key value
}
);
}
Packages
@plurid/deon-grammar • Visual Studio Code syntax highlighting
@plurid/deon-javascript • JavaScript / TypeScript implementation
@plurid/deon-rust • Rust implementation
