{"author":{"name":"Kaushal Modi","type":"card","url":"https://scripter.co/"},"children":[{"content":{"html":"\u003cdiv class=\"ox-hugo-toc toc\"\u003e\n\u003cdiv class=\"heading\"\u003eTable of Contents\u003c/div\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#my-flow-for-creating-this-library\"\u003eMy flow for creating this library\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#adapting-the-library-to-fit-ox-hugo\"\u003eAdapting the library to fit \u003ccode\u003eox-hugo\u003c/code\u003e\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#changes-in-ox-hugo-tests\"\u003eChanges in \u003ccode\u003eox-hugo\u003c/code\u003e tests\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#what-s-next\"\u003eWhat\u0026rsquo;s next?\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#unblocking-some-future-ox-hugo-improvements\"\u003eUnblocking some future \u003ccode\u003eox-hugo\u003c/code\u003e improvements\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/div\u003e\n\u003c!--endtoc--\u003e\n\u003cp\u003eIn my previous post \u003ca href=\"/defining-tomelr/\"\u003eDefining \u003cem\u003etomelr\u003c/em\u003e\u003c/a\u003e, I started toying with the idea\nof creating a library that would help convert any Lisp expression to a\nTOML config, and I share my vision (specification) of what this\nlibrary would look like.\u003c/p\u003e\n\u003cp\u003eI wasn\u0026rsquo;t even sure if I would be able to make the \u003ca href=\"https://github.com/kaushalmodi/tomelr\"\u003e\u003cstrong\u003etomelr\u003c/strong\u003e\u003c/a\u003e library\nfeature-complete at least to the extent of what \u003ccode\u003eox-hugo\u003c/code\u003e was already\ndoing! But to my surprise, the library development snowballed to a\ncompletion much earlier than I thought, and additionally it helped fix\nsome inconsistencies that the older TOML generation code had in\n\u003ccode\u003eox-hugo\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eIn this post, I start by (i) giving a broad overview of how the\ndevelopment of \u003ccode\u003etomelr\u003c/code\u003e happened, then (ii) briefly describe how it\ngot integrated into \u003ccode\u003eox-hugo\u003c/code\u003e, and finally (iii) how the use of this\nlibrary will unblock the path to addition of some cool features to\n\u003ccode\u003eox-hugo\u003c/code\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"my-flow-for-creating-this-library\"\u003eMy flow for creating this library\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#my-flow-for-creating-this-library\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003col\u003e\n\u003cli\u003eWrite the spec for the library.\n\u003cul\u003e\n\u003cli\u003eList all the formats of Lisp data I would expect it to process.\u003c/li\u003e\n\u003cli\u003eList the corresponding TOML data I would expect it to generate.\u003c/li\u003e\n\u003cli\u003eEnsure that I am not inventing my \u003cem\u003eown lisp syntax\u003c/em\u003e by confirming\nthat the expected TOML output matches the JSON generated from\nthat same lisp form (using the Emacs built-in \u003ccode\u003ejson.el\u003c/code\u003e library).\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eThat helped me write the tests first! \u0026ndash; \u003ca href=\"https://en.wikipedia.org/wiki/Test-driven_development\"\u003eTest Driven Development\n(TDD)\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eI started with writing tests for TOML booleans and then\nimplementing that (because that was the simplest and easiest). Of\ncourse, I used \u003ca href=\"/quick-intro-to-emacs-lisp-regression-testing/\"\u003eert\u003c/a\u003e for this! \u003ccode\u003eert\u003c/code\u003e helped me quickly create small\nmodular tests\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\n\u003ca href=\"https://github.com/kaushalmodi/tomelr/blob/867c82c9e230309c748de59bf8c0937b10d6fc64/test/tscalar.el#L31-L43\"\u003eHere\u0026rsquo;s\u003c/a\u003e the \u003cem\u003eert\u003c/em\u003e test for booleans as an example.\n\u003c/small\u003e\u003c/span\u003e\nand efficiently iterate through modifications in the library code\nuntil I got the tests to pass.\u003c/li\u003e\n\u003cli\u003eOnce that got working, I set up a continuous integration system\nusing \u003ca href=\"https://docs.github.com/en/actions\"\u003eGitHub Actions (GHA)\u003c/a\u003e. I used GHA because I host my library on\nGitHub. Also I already have a tried and tested setup that I could\nget up and going in a matter of few seconds. In general, this\nconcept would apply to any \u003cem\u003eContinuous Integration system\u003c/em\u003e. The CI\nsetup step should come early in the development of any project so\nthat incremental feature additions don\u0026rsquo;t start breaking previously\nadded features 😃.\u003c/li\u003e\n\u003cli\u003eThe library development just snowballed after this point .. added\nsupport for integers, floats, regular strings, multi-line strings,\narrays, TOML tables, arrays of TOML tables. By this time, the\nlibrary was about 80% finished.\u003c/li\u003e\n\u003cli\u003eThen came the difficult part .. stabilizing the library to support\nall the varieties of Lisp data I can think of. I must have put in\ndouble the time spent so far to finish the remaining 20% of the\nplanned features for this library 👉 \u003ca href=\"https://github.com/kaushalmodi/tomelr/tree/main/test\"\u003e\u003cstrong\u003etomelr test suite\u003c/strong\u003e\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eOnce I had the test suite complete and passing, it was time to do\nsome code cleanup:\n\u003cul\u003e\n\u003cli\u003eRemove duplicate code and break them off into smaller helper\nfunctions.\u003c/li\u003e\n\u003cli\u003eSee if the function defined in this library is already defined\nsomewhere else (in this case, I was able to use \u003ccode\u003ejson-plist-p\u003c/code\u003e\ndirectly from \u003ccode\u003ejson.el\u003c/code\u003e).\u003c/li\u003e\n\u003cli\u003eProof read the code.\u003c/li\u003e\n\u003cli\u003eProof read the docstrings and run \u003ccode\u003eM-x checkdoc\u003c/code\u003e to fix their\nformatting.\u003c/li\u003e\n\u003cli\u003eEnsure that the code compiles without any warnings.\u003c/li\u003e\n\u003cli\u003eRemove unnecessary customization options and case statements from\nthe library (while continuously ensuring that the \u003cem\u003eert\u003c/em\u003e tests\nstill pass).\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n\n\u003ch2 id=\"adapting-the-library-to-fit-ox-hugo\"\u003eAdapting the library to fit \u003ccode\u003eox-hugo\u003c/code\u003e\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#adapting-the-library-to-fit-ox-hugo\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eAfter polishing the library by its stand-alone testing, I decided to\nuse it with \u003ccode\u003eox-hugo\u003c/code\u003e and see how the test suite in that repo fared.\u003c/p\u003e\n\u003cp\u003eOf course I saw that a lot of tests failed now 😁.\u003c/p\u003e\n\u003cp\u003eThe main issue was that \u003ccode\u003etomelr\u003c/code\u003e was constructing multi-line strings\nsuch that the spaces translated exactly from Lisp data to TOML. So\n\u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nv\"\u003etomelr-encode\u003c/span\u003e \u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003efoo\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;line1\\nline2\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e)))\u003c/span\u003e\u003c/code\u003e would generate:\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"code-snippet--tomelr-mls-no-extra-ws\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-toml\" data-lang=\"toml\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nx\"\u003efoo\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u0026#34;\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003eline1\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003eline2\u0026#34;\u0026#34;\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"src-block-caption\"\u003e\n  \u003cspan class=\"src-block-number\"\u003e\u003ca href=\"#code-snippet--tomelr-mls-no-extra-ws\"\u003eCode Snippet 1\u003c/a\u003e:\u003c/span\u003e\n  Multi-line string with same white-space as in original data.. but not that \"pretty\"\n\u003c/div\u003e\n\u003cp\u003ewhereas \u003ccode\u003eox-hugo\u003c/code\u003e expected the same TOML to look like:\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"code-snippet--tomelr-pretty-mls\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-toml\" data-lang=\"toml\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nx\"\u003efoo\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u0026#34;\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003e  line1\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003e  line2\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003e  \u0026#34;\u0026#34;\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"src-block-caption\"\u003e\n  \u003cspan class=\"src-block-number\"\u003e\u003ca href=\"#code-snippet--tomelr-pretty-mls\"\u003eCode Snippet 2\u003c/a\u003e:\u003c/span\u003e\n  Pretty multi-line string, but with extra white-space\n\u003c/div\u003e\n\u003cp\u003eI had intentionally decided for \u003ccode\u003eox-hugo\u003c/code\u003e to have this latter format\nfor multi-line strings because (i) it made it more readable with the\n\u003cem\u003etriple-quotes\u003c/em\u003e out of the way on their own lines, (ii) the indented\nlines prevented the multi-line string from getting mixed with\nsurrounding TOML parameters, and \u003cstrong\u003emost importantly\u003c/strong\u003e (iii) these\nstrings were processed by the Hugo Markdown parser, and so it wasn\u0026rsquo;t\nsensitive to horizontal spaces.\u003c/p\u003e\n\u003cp\u003eAnd so the \u003ccode\u003etomelr-indent-multi-line-strings\u003c/code\u003e feature was born\n(\u003ca href=\"https://github.com/kaushalmodi/tomelr/commit/3362213172237f40ff0d9aa3ddf12b4bb00a3564\"\u003ecommit\u003c/a\u003e) which optionally made \u003ccode\u003etomelr\u003c/code\u003e export multi-line strings as\nexpected by \u003ccode\u003eox-hugo\u003c/code\u003e 😎.\u003c/p\u003e\n\n\u003ch2 id=\"changes-in-ox-hugo-tests\"\u003eChanges in \u003ccode\u003eox-hugo\u003c/code\u003e tests\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#changes-in-ox-hugo-tests\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eOnce I had finalized the integration of \u003ccode\u003etomelr\u003c/code\u003e into \u003ccode\u003eox-hugo\u003c/code\u003e, I had\nonly about 30 tests change out of roughly 400 tests. These changes\nwere welcome as they fixed all the inconsistencies in the older TOML\ngeneration code in \u003ccode\u003eox-hugo\u003c/code\u003e. If interested, you can see \u003ca href=\"https://github.com/kaushalmodi/ox-hugo/commit/c24ea792484598ffd2f8e786fadb823d48c8ec12\"\u003ethis commit\u003c/a\u003e\nfor the diff and details, but here\u0026rsquo;s the gist:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eNow \u003cem\u003enil\u003c/em\u003e value of a key in Lisp consistently implies that the key\nshould not be exported to TOML. So \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003efoo\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"no\"\u003enil\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e\u003c/code\u003e will result in \u003ccode\u003efoo\u003c/code\u003e \u003cstrong\u003enot\u003c/strong\u003e getting exported\nto TOML, whether that\u0026rsquo;s a top-level key or a key in a nested TOML\nmap or array. If you need to set a key to a boolean \u003cem\u003efalse\u003c/em\u003e, use\n\u003ccode\u003e\u0026quot;false\u0026quot;\u003c/code\u003e or any value from \u003ccode\u003etomelr-false\u003c/code\u003e.\u003c/li\u003e\n\u003cli\u003eEarlier \u003cem\u003eempty string\u003c/em\u003e value as in \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003efoo\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e\u003c/code\u003e behaved like the current \u003cem\u003enil\u003c/em\u003e\nimplementation. That\u0026rsquo;s not the case any more. Now that empty string\nwill export as \u003ccode class=\"code-inline language-toml\"\u003e\u003cspan class=\"nx\"\u003efoo\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u003c/span\u003e\u003c/code\u003e in TOML.\u003c/li\u003e\n\u003cli\u003eNow if a string has a quote character (\u003ccode\u003e\u0026quot;\u003c/code\u003e) in it, that value will\nauto-export as TOML multi-line string. I like the readability of\nthis more than that of backslash-escaped double-quotes.\u003c/li\u003e\n\u003cli\u003eNow the nested tables like \u003ccode class=\"code-inline language-toml\"\u003e\u003cspan class=\"p\"\u003e[\u003c/span\u003e\u003cspan class=\"nx\"\u003emenu\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"s2\"\u003e\u0026#34;nested menu\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003c/code\u003e export with their parent table keys like \u003ccode class=\"code-inline language-toml\"\u003e\u003cspan class=\"p\"\u003e[\u003c/span\u003e\u003cspan class=\"nx\"\u003emenu\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003c/code\u003e. As per the TOML spec, this is not required. But now\nthat \u003ccode\u003etomelr\u003c/code\u003e has added a generic support for any TOML table, this\nchange happens as a result of consistency 💯.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eIn summary, the changes in \u003ccode\u003eox-hugo\u003c/code\u003e TOML front-matter exports were\nmostly cosmetic, and if they were not cosmetic, they were consistency\nfixes.\u003c/p\u003e\n\n\u003ch2 id=\"what-s-next\"\u003eWhat\u0026rsquo;s next?\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#what-s-next\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cdl\u003e\n\u003cdt\u003etomelr\u003c/dt\u003e\n\u003cdd\u003eThe library is pretty much feature complete ✨ as\nmany of the examples from \u003ca href=\"https://toml.io/en/v1.0.0\"\u003eTOML v1.0.0 spec\u003c/a\u003e have been added to its\ntest suite, and .. it is supporting all the \u003ccode\u003eox-hugo\u003c/code\u003e use cases.\n\u003cp\u003eThe library though has \u003ca href=\"https://github.com/kaushalmodi/tomelr#limitations\"\u003eone limitation\u003c/a\u003e that I\u0026rsquo;d like to resolve at\nsome point \u0026mdash; Right now, we require the Lisp data to first list all\nthe scalar keys and then list the TOML tables and arrays of\ntables. But at the moment, I don\u0026rsquo;t know how to fix that, and also\n\u003ccode\u003eox-hugo\u003c/code\u003e is not affected by that (because it already populates the\nfront-matter alist in the correct order). So fixing this is not\nurgent, but of course, if someone can help me out with that, I\u0026rsquo;d\nwelcome that! 🙏.\u003c/p\u003e\n\u003c/dd\u003e\n\u003cdt\u003eox-hugo\u003c/dt\u003e\n\u003cdd\u003eGiven that \u003ccode\u003etomelr\u003c/code\u003e allows robustly exporting any Lisp\ndata expression to TOML, I do not see any value in continuing with\nYAML generation support using the old custom code.\n\u003cp\u003e📢 In near future, I plan to get rid of the\n\u003ccode\u003eorg-hugo-front-matter-format\u003c/code\u003e customization variable from \u003ccode\u003eox-hugo\u003c/code\u003e\n\u0026mdash; thus deprecating YAML export support\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nThis change should not functionally affect the YAML front-matter\nfans out there because the front-matter that \u003ccode\u003eox-hugo\u003c/code\u003e is exporting\nis mainly for Hugo\u0026rsquo;s consumption. The only scenario where I see that\nthis change can be breaking is if the user is using YAML format\n\u003cem\u003eextra front-matter\u003c/em\u003e blocks. If so, unfortunately, they will need to\nconvert those to TOML manually.\n\u003c/small\u003e\u003c/span\u003e\nand sticking with using just TOML for the front-matter.\u003c/p\u003e\n\u003c/dd\u003e\n\u003c/dl\u003e\n\n\u003ch3 id=\"unblocking-some-future-ox-hugo-improvements\"\u003eUnblocking some future \u003ccode\u003eox-hugo\u003c/code\u003e improvements\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#unblocking-some-future-ox-hugo-improvements\"\u003e#\u003c/a\u003e\u003c/h3\u003e\n\n\n\u003cp\u003eThis decision will open up the doors to add more features to \u003ccode\u003eox-hugo\u003c/code\u003e\nlike:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\n\u003cp\u003eExporting Org \u003ccode\u003e:LOGBOOK:\u003c/code\u003e drawers to TOML front-matter (\u003ca href=\"https://github.com/kaushalmodi/ox-hugo/pull/504\"\u003eox-hugo #\n\u003cstrong\u003e504\u003c/strong\u003e\u003c/a\u003e)\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eExporting Org Special Blocks to user-configurable front-matter\n(\u003ca href=\"https://github.com/kaushalmodi/ox-hugo/pull/627\"\u003eox-hugo # \u003cstrong\u003e627\u003c/strong\u003e\u003c/a\u003e)\u003c/p\u003e\n\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eSupporting more complex data in Lisp form using\n\u003ccode\u003e:EXPORT_HUGO_CUSTOM_FRONT_MATTER:\u003c/code\u003e which could translate to nested\nTOML tables or arrays of TOML tables.\u003c/p\u003e\n\u003cp\u003eFinally, there won\u0026rsquo;t be a need to use the \u003ca href=\"https://ox-hugo.scripter.co/doc/custom-front-matter/#front-matter-extra\"\u003e\u0026ldquo;Extra front-matter\u0026rdquo;\u003c/a\u003e\nworkaround. For example, it would be possible to represent the data\nin that first example on that page as \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"nb\"\u003e:foo\u003c/span\u003e \u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nb\"\u003e:bar\u003c/span\u003e \u003cspan class=\"mi\"\u003e1\u003c/span\u003e \u003cspan class=\"nb\"\u003e:zoo\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;abc\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nb\"\u003e:bar\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e \u003cspan class=\"nb\"\u003e:zoo\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;def\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e\u003c/code\u003e in the\n\u003ccode\u003e:EXPORT_HUGO_CUSTOM_FRONT_MATTER:\u003c/code\u003e property.\u003c/p\u003e\n\u003c/li\u003e\n\u003c/ol\u003e\n","text":" Table of Contents My flow for creating this library Adapting the library to fit ox-hugo Changes in ox-hugo tests What\u0026rsquo;s next? Unblocking some future ox-hugo improvements In my previous post Defining tomelr, I started toying with the idea of creating a library that would help convert any Lisp expression to a TOML config, and I share my vision (specification) of what this library would look like.\nI wasn\u0026rsquo;t even sure if I would be able to make the tomelr library feature-complete at least to the extent of what ox-hugo was already doing! But to my surprise, the library development snowballed to a completion much earlier than I thought, and additionally it helped fix some inconsistencies that the older TOML generation code had in ox-hugo.\nIn this post, I start by (i) giving a broad overview of how the development of tomelr happened, then (ii) briefly describe how it got integrated into ox-hugo, and finally (iii) how the use of this library will unblock the path to addition of some cool features to ox-hugo.\nMy flow for creating this library\u0026nbsp;# Write the spec for the library. List all the formats of Lisp data I would expect it to process. List the corresponding TOML data I would expect it to generate. Ensure that I am not inventing my own lisp syntax by confirming that the expected TOML output matches the JSON generated from that same lisp form (using the Emacs built-in json.el library). That helped me write the tests first! \u0026ndash; Test Driven Development (TDD). I started with writing tests for TOML booleans and then implementing that (because that was the simplest and easiest). Of course, I used ert for this! ert helped me quickly create small modular tests Here\u0026rsquo;s the ert test for booleans as an example. and efficiently iterate through modifications in the library code until I got the tests to pass. Once that got working, I set up a continuous integration system using GitHub Actions (GHA). I used GHA because I host my library on GitHub. Also I already have a tried and tested setup that I could get up and going in a matter of few seconds. In general, this concept would apply to any Continuous Integration system. The CI setup step should come early in the development of any project so that incremental feature additions don\u0026rsquo;t start breaking previously added features 😃. The library development just snowballed after this point .. added support for integers, floats, regular strings, multi-line strings, arrays, TOML tables, arrays of TOML tables. By this time, the library was about 80% finished. Then came the difficult part .. stabilizing the library to support all the varieties of Lisp data I can think of. I must have put in double the time spent so far to finish the remaining 20% of the planned features for this library 👉 tomelr test suite Once I had the test suite complete and passing, it was time to do some code cleanup: Remove duplicate code and break them off into smaller helper functions. See if the function defined in this library is already defined somewhere else (in this case, I was able to use json-plist-p directly from json.el). Proof read the code. Proof read the docstrings and run M-x checkdoc to fix their formatting. Ensure that the code compiles without any warnings. Remove unnecessary customization options and case statements from the library (while continuously ensuring that the ert tests still pass). Adapting the library to fit ox-hugo\u0026nbsp;# After polishing the library by its stand-alone testing, I decided to use it with ox-hugo and see how the test suite in that repo fared.\nOf course I saw that a lot of tests failed now 😁.\nThe main issue was that tomelr was constructing multi-line strings such that the spaces translated exactly from Lisp data to TOML. So (tomelr-encode \u0026#39;((foo . \u0026#34;line1\\nline2\u0026#34;))) would generate:\nfoo = \u0026#34;\u0026#34;\u0026#34; line1 line2\u0026#34;\u0026#34;\u0026#34; Code Snippet 1: Multi-line string with same white-space as in original data.. but not that \"pretty\" whereas ox-hugo expected the same TOML to look like:\nfoo = \u0026#34;\u0026#34;\u0026#34; line1 line2 \u0026#34;\u0026#34;\u0026#34; Code Snippet 2: Pretty multi-line string, but with extra white-space I had intentionally decided for ox-hugo to have this latter format for multi-line strings because (i) it made it more readable with the triple-quotes out of the way on their own lines, (ii) the indented lines prevented the multi-line string from getting mixed with surrounding TOML parameters, and most importantly (iii) these strings were processed by the Hugo Markdown parser, and so it wasn\u0026rsquo;t sensitive to horizontal spaces.\nAnd so the tomelr-indent-multi-line-strings feature was born (commit) which optionally made tomelr export multi-line strings as expected by ox-hugo 😎.\nChanges in ox-hugo tests\u0026nbsp;# Once I had finalized the integration of tomelr into ox-hugo, I had only about 30 tests change out of roughly 400 tests. These changes were welcome as they fixed all the inconsistencies in the older TOML generation code in ox-hugo. If interested, you can see this commit for the diff and details, but here\u0026rsquo;s the gist:\nNow nil value of a key in Lisp consistently implies that the key should not be exported to TOML. So \u0026#39;((foo . nil)) will result in foo not getting exported to TOML, whether that\u0026rsquo;s a top-level key or a key in a nested TOML map or array. If you need to set a key to a boolean false, use \u0026quot;false\u0026quot; or any value from tomelr-false. Earlier empty string value as in \u0026#39;((foo . \u0026#34;\u0026#34;)) behaved like the current nil implementation. That\u0026rsquo;s not the case any more. Now that empty string will export as foo = \u0026#34;\u0026#34; in TOML. Now if a string has a quote character (\u0026quot;) in it, that value will auto-export as TOML multi-line string. I like the readability of this more than that of backslash-escaped double-quotes. Now the nested tables like [menu.\u0026#34;nested menu\u0026#34;] export with their parent table keys like [menu]. As per the TOML spec, this is not required. But now that tomelr has added a generic support for any TOML table, this change happens as a result of consistency 💯. In summary, the changes in ox-hugo TOML front-matter exports were mostly cosmetic, and if they were not cosmetic, they were consistency fixes.\nWhat\u0026rsquo;s next?\u0026nbsp;# tomelr The library is pretty much feature complete ✨ as many of the examples from TOML v1.0.0 spec have been added to its test suite, and .. it is supporting all the ox-hugo use cases. The library though has one limitation that I\u0026rsquo;d like to resolve at some point \u0026mdash; Right now, we require the Lisp data to first list all the scalar keys and then list the TOML tables and arrays of tables. But at the moment, I don\u0026rsquo;t know how to fix that, and also ox-hugo is not affected by that (because it already populates the front-matter alist in the correct order). So fixing this is not urgent, but of course, if someone can help me out with that, I\u0026rsquo;d welcome that! 🙏.\nox-hugo Given that tomelr allows robustly exporting any Lisp data expression to TOML, I do not see any value in continuing with YAML generation support using the old custom code. 📢 In near future, I plan to get rid of the org-hugo-front-matter-format customization variable from ox-hugo \u0026mdash; thus deprecating YAML export support This change should not functionally affect the YAML front-matter fans out there because the front-matter that ox-hugo is exporting is mainly for Hugo\u0026rsquo;s consumption. The only scenario where I see that this change can be breaking is if the user is using YAML format extra front-matter blocks. If so, unfortunately, they will need to convert those to TOML manually. and sticking with using just TOML for the front-matter.\nUnblocking some future ox-hugo improvements\u0026nbsp;# This decision will open up the doors to add more features to ox-hugo like:\nExporting Org :LOGBOOK: drawers to TOML front-matter (ox-hugo # 504)\nExporting Org Special Blocks to user-configurable front-matter (ox-hugo # 627)\nSupporting more complex data in Lisp form using :EXPORT_HUGO_CUSTOM_FRONT_MATTER: which could translate to nested TOML tables or arrays of TOML tables.\nFinally, there won\u0026rsquo;t be a need to use the \u0026ldquo;Extra front-matter\u0026rdquo; workaround. For example, it would be possible to represent the data in that first example on that page as :foo ((:bar 1 :zoo \u0026#34;abc\u0026#34;) (:bar 2 :zoo \u0026#34;def\u0026#34;)) in the :EXPORT_HUGO_CUSTOM_FRONT_MATTER: property.\n"},"name":"Presenting tomelr!","published":"2022-05-04T01:03:00-04:00","summary":"In this post, I introduce a little library I created for ox-hugo to have a robust mechanism for generating TOML from any Lisp expression.","type":"entry","url":"https://scripter.co/presenting-tomelr/"},{"content":{"html":"\u003cdiv class=\"ox-hugo-toc toc\"\u003e\n\u003cdiv class=\"heading\"\u003eTable of Contents\u003c/div\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#using-json-encode-as-reference\"\u003eUsing \u003ccode\u003ejson-encode\u003c/code\u003e as reference\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#mapping-scalar-data-to-toml\"\u003eMapping scalar data to TOML\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#about-false\"\u003eAbout \u003ccode\u003e:false\u003c/code\u003e\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#mapping-lists-to-toml\"\u003eMapping lists to TOML\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#mapping-lists-of-lists-to-toml\"\u003eMapping lists of lists to TOML\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#mapping-other-object-types\"\u003eMapping other object types\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#closing\"\u003eClosing\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/div\u003e\n\u003c!--endtoc--\u003e\n\u003cp\u003e\u003ccode\u003eox-hugo\u003c/code\u003e has some custom code that generates \u003ca href=\"https://toml.io/en/\"\u003eTOML\u003c/a\u003e\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nI ❤️ TOML. As the makers of this config format put it.. \u0026ldquo;it\u0026rsquo;s a\nformat \u003cem\u003efor humans\u003c/em\u003e\u0026rdquo;! \u0026mdash; No need to deal with indentations, weird\nsyntax for multi-line strings, no prohibition on adding comments to\nthe config, or dealing with careful placement of commas and braces.\n\u003c/small\u003e\u003c/span\u003e\nfor \u003ca href=\"https://gohugo.io/\"\u003eHugo\u003c/a\u003e front-matter, based on the Org keywords and other meta-data\nthat the user sets in their Org file. But this TOML generation code is\nnot generic enough for any TOML object type.\u003c/p\u003e\n\u003cp\u003eAs I am \u003ca href=\"https://github.com/kaushalmodi/ox-hugo/pull/504#issuecomment-1093592230\"\u003ewriting up a definition\u003c/a\u003e on how to export \u003ccode\u003e:LOGBOOK:\u003c/code\u003e \u003ca href=\"https://orgmode.org/manual/Drawers.html\" title=\"Emacs Lisp: (info \u0026quot;(org) Drawers\u0026quot;)\"\u003edrawers\u003c/a\u003e,\nI felt it\u0026rsquo;s a good time to polish the whole \u003cstrong\u003eLisp data → TOML\u003c/strong\u003e\nconversion code, and may be package that into a separate library.\u003c/p\u003e\n\u003cdiv class=\"verse\"\u003e\n\u003cp\u003e        It\u0026rsquo;s kind of an ambitious project \u0026mdash; I am calling it \u003ca href=\"https://github.com/kaushalmodi/tomelr\"\u003etom​\u003cstrong\u003eel\u003c/strong\u003e​r\u003c/a\u003e ✨\u003cbr /\u003e\u003c/p\u003e\n\u003c/div\u003e\n\u003cp\u003eIt\u0026rsquo;s a big undertaking to create a generic library for this kind of\ndata format conversion. But even before I start coding or think if I\ncan complete this project, I need to \u003cem\u003espec\u003c/em\u003e\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nI am not sure if it\u0026rsquo;s a widely used verb, but \u003cem\u003eto spec\u003c/em\u003e means \u003cem\u003eto\nwrite a specification for something\u003c/em\u003e.\n\u003c/small\u003e\u003c/span\u003e\nit. I need to understand early-on how the \u003cem\u003eS-exp\u003c/em\u003e (Symbolic lisp\nexpression) would needed to look for each kind of generated TOML\nobject \u0026mdash; scalars, lists, lists of lists, maps, lists of maps, etc.\u003c/p\u003e\n\n\u003ch2 id=\"using-json-encode-as-reference\"\u003eUsing \u003ccode\u003ejson-encode\u003c/code\u003e as reference\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#using-json-encode-as-reference\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eThe aim of the \u003ccode\u003etomelr\u003c/code\u003e library is to take some \u003ccode\u003e(lisp data)\u003c/code\u003e and\nconvert that TOML. But I did not want to invent my own \u003cem\u003elisp data\nconvention\u003c/em\u003e for this! So I decided to stick with the lisp expression\nconventions understood by the \u003ccode\u003ejson-encode\u003c/code\u003e function from the Emacs\ncore library \u003ccode\u003ejson.el\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eCredit for the \u003ccode\u003ejson.el\u003c/code\u003e idea goes to \u003ca href=\"https://twitter.com/pdcawley/status/1519007598896369664\"\u003ethis tweet\u003c/a\u003e by \u003ca href=\"https://twitter.com/pdcawley\"\u003ePiers Cawley\u003c/a\u003e:\u003c/p\u003e\n\u003cblockquote\u003e\n\u003cp\u003eI\u0026rsquo;d suggest that\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nBy \u0026ldquo;that\u0026rdquo;, he\u0026rsquo;s referring to adding support for exporting front-matter\nto JSON in \u003ccode\u003eox-hugo\u003c/code\u003e.\n\u003c/small\u003e\u003c/span\u003e\n, since emacs has built in JSON support these days, you\ndon\u0026rsquo;t really have to worry about the commas and braces, just build the\ns-exp you want and export, but it\u0026rsquo;s you that\u0026rsquo;s writing the code and\nI\u0026rsquo;m just delighted that it exists.\u003c/p\u003e\n\u003cp\u003eThank you for your efforts.\u003c/p\u003e\n\u003c/blockquote\u003e\n\u003cp\u003eI am not sold on adding support of yet another front-matter format to\n\u003ccode\u003eox-hugo\u003c/code\u003e. I might not use \u003ccode\u003ejson.el\u003c/code\u003e for that, but it definitely\nhelped me a lot with coming up with this library\u0026rsquo;s spec 😆.\u003c/p\u003e\n\n\u003ch2 id=\"mapping-scalar-data-to-toml\"\u003eMapping scalar data to TOML\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#mapping-scalar-data-to-toml\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eFiguring out the Lisp representation for scalar (plain key-value\npairs) TOML objects was easy. \u003ccode\u003ejson.el\u003c/code\u003e helped figure out how to deal\nwith \u003cem\u003enil\u003c/em\u003e and \u003cem\u003efalse\u003c/em\u003e values.\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"table--mapping-scalar-lisp-to-toml\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"table-caption\"\u003e\n  \u003cspan class=\"table-number\"\u003e\u003ca href=\"#table--mapping-scalar-lisp-to-toml\"\u003eTable 1\u003c/a\u003e:\u003c/span\u003e\n  Mapping of \u003ci\u003escalar\u003c/i\u003e Lisp data to TOML\n\u003c/div\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eLisp S-exp\u003c/th\u003e\n\u003cth\u003eTOML\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((int_key . 123))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003eint_key = 123\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((float_key . 1.23))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003efloat_key = 1.23\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((date_key . 2022-04-27))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003edate_key = 2022-04-27\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((bool_key . t))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ebool_key = true\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((any_key . nil))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cem\u003e(key removed in TOML)\u003c/em\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((bool_key . :false))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003ebool_key = false\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003ch3 id=\"about-false\"\u003eAbout \u003ccode\u003e:false\u003c/code\u003e\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#about-false\"\u003e#\u003c/a\u003e\u003c/h3\u003e\n\n\n\u003cp\u003e\u003ccode\u003ejson.el\u003c/code\u003e defines a variable \u003ccode\u003ejson-false\u003c/code\u003e that\u0026rsquo;s set to the value\n\u003ccode\u003e:json-false\u003c/code\u003e. This is because in JSON, the \u003cem\u003enull\u003c/em\u003e value is different\nfrom the boolean \u003ccode\u003efalse\u003c/code\u003e value.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cem\u003enil\u003c/em\u003e in Lisp → \u003cem\u003enull\u003c/em\u003e in JSON\u003c/li\u003e\n\u003cli\u003e\u003ccode\u003e:json-false\u003c/code\u003e in Lisp → \u003ccode\u003efalse\u003c/code\u003e in JSON\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eInspired by that decision of \u003ccode\u003ejson.el\u003c/code\u003e, I am thinking of using\n\u003ccode\u003e:false\u003c/code\u003e as the special value that will set the equivalent TOML value\nto boolean \u003ccode\u003efalse\u003c/code\u003e.\u003c/p\u003e\n\u003cdiv class=\"note\"\u003e\n\u003cp\u003eTOML does not define a \u003cem\u003enull\u003c/em\u003e value, but if the Lisp value is \u003cem\u003enil\u003c/em\u003e,\nthat key will simply not be translated to TOML.\u003c/p\u003e\n\u003c/div\u003e\n\n\u003ch2 id=\"mapping-lists-to-toml\"\u003eMapping lists to TOML\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#mapping-lists-to-toml\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eMapping lists was simple.. because in Lisp, a list value of course\nlooks like \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003efoo\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e \u003cspan class=\"mi\"\u003e3\u003c/span\u003e \u003cspan class=\"mi\"\u003e4\u003c/span\u003e \u003cspan class=\"mi\"\u003e5\u003c/span\u003e\u003cspan class=\"p\"\u003e)))\u003c/span\u003e\u003c/code\u003e\n😃.\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"table--mapping-list-lisp-to-toml\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"table-caption\"\u003e\n  \u003cspan class=\"table-number\"\u003e\u003ca href=\"#table--mapping-list-lisp-to-toml\"\u003eTable 2\u003c/a\u003e:\u003c/span\u003e\n  Mapping of \u003ci\u003elist\u003c/i\u003e Lisp data to TOML\n\u003c/div\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eLisp S-exp\u003c/th\u003e\n\u003cth\u003eTOML\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((list_key1 . (1 2 3)))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003elist_key1 = [1, 2, 3]\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((list_key2 . (\u0026quot;a\u0026quot; \u0026quot;b\u0026quot; \u0026quot;c\u0026quot;)))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003elist_key2 = [\u0026quot;a\u0026quot;, \u0026quot;b\u0026quot;, \u0026quot;c\u0026quot;]\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003ch2 id=\"mapping-lists-of-lists-to-toml\"\u003eMapping lists of lists to TOML\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#mapping-lists-of-lists-to-toml\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003e\u003ca id=\"table--mapping-list-of-list-lisp-to-toml\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"table-caption\"\u003e\n  \u003cspan class=\"table-number\"\u003e\u003ca href=\"#table--mapping-list-of-list-lisp-to-toml\"\u003eTable 3\u003c/a\u003e:\u003c/span\u003e\n  Mapping of \u003ci\u003elist of list\u003c/i\u003e Lisp data to TOML\n\u003c/div\u003e\n\u003ctable\u003e\n\u003cthead\u003e\n\u003ctr\u003e\n\u003cth\u003eLisp S-exp\u003c/th\u003e\n\u003cth\u003eTOML\u003c/th\u003e\n\u003c/tr\u003e\n\u003c/thead\u003e\n\u003ctbody\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003ccode\u003e'((lol_key . [(1 2) (3 4)]))\u003c/code\u003e\u003c/td\u003e\n\u003ctd\u003e\u003ccode\u003elol_key = [ [1, 2], [3, 4] ]\u003c/code\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eI was going to use \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003elol_key\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e \u003cspan class=\"mi\"\u003e4\u003c/span\u003e\u003cspan class=\"p\"\u003e))))\u003c/span\u003e\u003c/code\u003e as the reference Lisp expression for \u003ccode class=\"code-inline language-toml\"\u003e\u003cspan class=\"nx\"\u003elol_key\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"p\"\u003e[\u003c/span\u003e \u003cspan class=\"p\"\u003e[\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e],\u003c/span\u003e \u003cspan class=\"p\"\u003e[\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e \u003cspan class=\"mi\"\u003e4\u003c/span\u003e\u003cspan class=\"p\"\u003e]\u003c/span\u003e \u003cspan class=\"p\"\u003e]\u003c/span\u003e\u003c/code\u003e. But I found out that\n\u003ccode\u003ejson-encode\u003c/code\u003e throws an error if you pass it that expression! I don\u0026rsquo;t\nunderstand the reason for that error, and so I have \u003ca href=\"https://lists.gnu.org/r/help-gnu-emacs/2022-04/msg00240.html\"\u003easked for help\u003c/a\u003e on\nthe \u003cem\u003ehelp-gnu-emacs\u003c/em\u003e mailing list.\u003c/p\u003e\n\u003cp\u003eBut while that question gets resolved, I wanted to move forward with\nthe spec definition. After some trial-and-error and\nreverse-engineering \u003ccode\u003ejson.el\u003c/code\u003e\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nI knew how I wanted TOML to look. So I used \u003ca href=\"https://toolkit.site/format.html\"\u003ean online JSON/TOML\nconverter\u003c/a\u003e to convert that TOML snippet to JSON, and then used\n\u003ccode\u003ejson-read\u003c/code\u003e to convert JSON to Lisp expression.\n\u003c/small\u003e\u003c/span\u003e\n, I learned that \u003cem\u003elist of list\u003c/em\u003e data needs to be represented using a \u003ca href=\"https://www.gnu.org/software/emacs/manual/html_node/elisp/Vector-Type.html\" title=\"Emacs Lisp: (info \u0026quot;(elisp) Vector Type\u0026quot;)\"\u003eVector type\u003c/a\u003e, and so \u003ccode class=\"code-inline language-emacs-lisp\"\u003e\u003cspan class=\"o\"\u003e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003elol_key\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"p\"\u003e[(\u003c/span\u003e\u003cspan class=\"mi\"\u003e1\u003c/span\u003e \u003cspan class=\"mi\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"mi\"\u003e3\u003c/span\u003e \u003cspan class=\"mi\"\u003e4\u003c/span\u003e\u003cspan class=\"p\"\u003e)]))\u003c/span\u003e\u003c/code\u003e would be the correct expression \u0026ndash; \u003cem\u003eNotice the use\nof square brackets instead of parentheses for the outer vector\u003c/em\u003e.\u003c/p\u003e\n\n\u003ch2 id=\"mapping-other-object-types\"\u003eMapping other object types\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#mapping-other-object-types\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eOnce I figured out how to map the above data types, mapping Lisp data\nto \u003cem\u003eTOML Tables\u003c/em\u003e aka \u003cem\u003edictionaries\u003c/em\u003e and \u003cem\u003earrays of Tables\u003c/em\u003e was a\nbreeze\n\u003cspan class=\"sidenote-number\"\u003e\u003csmall class=\"sidenote\"\u003e\nOf course, the \u003cem\u003ebreeze\u003c/em\u003e is referring to the ease of writing the spec\nfor these 😆. Implementation-wise, the \u003cem\u003etables\u003c/em\u003e, \u003cem\u003earrays of\ntables\u003c/em\u003e, and the especially \u003cstrong\u003enested\u003c/strong\u003e variants of those are going to be\nthe most challenging.\n\u003c/small\u003e\u003c/span\u003e\n.\u003c/p\u003e\n\n\u003ch2 id=\"closing\"\u003eClosing\u0026nbsp;\u003ca class=\"headline-hash no-text-decoration\" href=\"#closing\"\u003e#\u003c/a\u003e\u003c/h2\u003e\n\n\n\u003cp\u003eIt was fun coming up with an initial draft of the specification for\nthis library. My next steps would be to gradually add TOML generator\nfunctions (as I find time) to this library, along with \u003ca href=\"/quick-intro-to-emacs-lisp-regression-testing/\"\u003eERT tests\u003c/a\u003e!\nEventually, I will remove the existing TOML generation code from\n\u003ccode\u003eox-hugo\u003c/code\u003e and depend on this library.\u003c/p\u003e\n\u003cp\u003eGetting back to \u003ca href=\"https://github.com/kaushalmodi/ox-hugo/pull/504#issuecomment-1093592230\"\u003emy plan\u003c/a\u003e for exporting \u003ccode\u003e:LOGBOOK:\u003c/code\u003e drawers in\n\u003ccode\u003eox-hugo\u003c/code\u003e, based on this spec, I will need to construct this date in\nEmacs-Lisp:\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"code-snippet--org-logbook-lisp\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-emacs-lisp\" data-lang=\"emacs-lisp\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nv\"\u003eorg_logbook\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"p\"\u003e(((\u003c/span\u003e\u003cspan class=\"nv\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"nv\"\u003e2022-04-08T14:53:00-04:00\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                 \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nv\"\u003enote\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;This note addition prompt shows up on typing the `C-c C-z` binding.\\nSee [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"nv\"\u003e2018-09-06T11:45:00-04:00\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                 \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nv\"\u003enote\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;Another note **bold** _italics_.\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e))\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                \u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"nv\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"nv\"\u003e2018-09-06T11:37:00-04:00\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e                 \u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"nv\"\u003enote\u003c/span\u003e \u003cspan class=\"o\"\u003e.\u003c/span\u003e \u003cspan class=\"s\"\u003e\u0026#34;A note `mono`.\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e))))\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"src-block-caption\"\u003e\n  \u003cspan class=\"src-block-number\"\u003e\u003ca href=\"#code-snippet--org-logbook-lisp\"\u003eCode Snippet 1\u003c/a\u003e:\u003c/span\u003e\n  Example data from a \u003ccode\u003e:LOGBOOK:\u003c/code\u003e drawer in Lisp format\n\u003c/div\u003e\n\u003cp\u003e.. will translate to this in TOML:\u003c/p\u003e\n\u003cp\u003e\u003ca id=\"code-snippet--org-logbook-toml\"\u003e\u003c/a\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-toml\" data-lang=\"toml\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e[[\u003c/span\u003e\u003cspan class=\"nx\"\u003eorg_logbook\u003c/span\u003e\u003cspan class=\"p\"\u003e]]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"ld\"\u003e2022-04-08T14:53:00-04:00\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003enote\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u0026#34;This note addition prompt shows up on typing the `C-c C-z` binding.\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"s2\"\u003eSee [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).\u0026#34;\u0026#34;\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e[[\u003c/span\u003e\u003cspan class=\"nx\"\u003eorg_logbook\u003c/span\u003e\u003cspan class=\"p\"\u003e]]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"ld\"\u003e2018-09-06T11:45:00-04:00\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003enote\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u0026#34;Another note **bold** _italics_.\u0026#34;\u0026#34;\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e[[\u003c/span\u003e\u003cspan class=\"nx\"\u003eorg_logbook\u003c/span\u003e\u003cspan class=\"p\"\u003e]]\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003etimestamp\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"ld\"\u003e2018-09-06T11:37:00-04:00\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e  \u003cspan class=\"nx\"\u003enote\u003c/span\u003e \u003cspan class=\"p\"\u003e=\u003c/span\u003e \u003cspan class=\"s2\"\u003e\u0026#34;\u0026#34;\u0026#34;A note `mono`.\u0026#34;\u0026#34;\u0026#34;\u003c/span\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cdiv class=\"src-block-caption\"\u003e\n  \u003cspan class=\"src-block-number\"\u003e\u003ca href=\"#code-snippet--org-logbook-toml\"\u003eCode Snippet 2\u003c/a\u003e:\u003c/span\u003e\n  Same example data from the \u003ccode\u003e:LOGBOOK:\u003c/code\u003e drawer translated to TOML format\n\u003c/div\u003e\n\u003chr\u003e\n\u003cp\u003eCheck out the below link where I have documented the equivalent\nexpressions between Lisp, TOML and JSON for all the object types.\u003c/p\u003e\n\u003cp\u003e👉 \u003ca href=\"https://github.com/kaushalmodi/tomelr/blob/main/README.org\"\u003etom​\u003cstrong\u003eel\u003c/strong\u003e​r Specification\u003c/a\u003e\u003c/p\u003e\n","text":" Table of Contents Using json-encode as reference Mapping scalar data to TOML About :false Mapping lists to TOML Mapping lists of lists to TOML Mapping other object types Closing ox-hugo has some custom code that generates TOML I ❤️ TOML. As the makers of this config format put it.. \u0026ldquo;it\u0026rsquo;s a format for humans\u0026rdquo;! \u0026mdash; No need to deal with indentations, weird syntax for multi-line strings, no prohibition on adding comments to the config, or dealing with careful placement of commas and braces. for Hugo front-matter, based on the Org keywords and other meta-data that the user sets in their Org file. But this TOML generation code is not generic enough for any TOML object type.\nAs I am writing up a definition on how to export :LOGBOOK: drawers, I felt it\u0026rsquo;s a good time to polish the whole Lisp data → TOML conversion code, and may be package that into a separate library.\nIt\u0026rsquo;s kind of an ambitious project \u0026mdash; I am calling it tom​el​r ✨\nIt\u0026rsquo;s a big undertaking to create a generic library for this kind of data format conversion. But even before I start coding or think if I can complete this project, I need to spec I am not sure if it\u0026rsquo;s a widely used verb, but to spec means to write a specification for something. it. I need to understand early-on how the S-exp (Symbolic lisp expression) would needed to look for each kind of generated TOML object \u0026mdash; scalars, lists, lists of lists, maps, lists of maps, etc.\nUsing json-encode as reference\u0026nbsp;# The aim of the tomelr library is to take some (lisp data) and convert that TOML. But I did not want to invent my own lisp data convention for this! So I decided to stick with the lisp expression conventions understood by the json-encode function from the Emacs core library json.el.\nCredit for the json.el idea goes to this tweet by Piers Cawley:\nI\u0026rsquo;d suggest that By \u0026ldquo;that\u0026rdquo;, he\u0026rsquo;s referring to adding support for exporting front-matter to JSON in ox-hugo. , since emacs has built in JSON support these days, you don\u0026rsquo;t really have to worry about the commas and braces, just build the s-exp you want and export, but it\u0026rsquo;s you that\u0026rsquo;s writing the code and I\u0026rsquo;m just delighted that it exists.\nThank you for your efforts.\nI am not sold on adding support of yet another front-matter format to ox-hugo. I might not use json.el for that, but it definitely helped me a lot with coming up with this library\u0026rsquo;s spec 😆.\nMapping scalar data to TOML\u0026nbsp;# Figuring out the Lisp representation for scalar (plain key-value pairs) TOML objects was easy. json.el helped figure out how to deal with nil and false values.\nTable 1: Mapping of scalar Lisp data to TOML Lisp S-exp TOML '((int_key . 123)) int_key = 123 '((float_key . 1.23)) float_key = 1.23 '((date_key . 2022-04-27)) date_key = 2022-04-27 '((bool_key . t)) bool_key = true '((any_key . nil)) (key removed in TOML) '((bool_key . :false)) bool_key = false About :false\u0026nbsp;# json.el defines a variable json-false that\u0026rsquo;s set to the value :json-false. This is because in JSON, the null value is different from the boolean false value.\nnil in Lisp → null in JSON :json-false in Lisp → false in JSON Inspired by that decision of json.el, I am thinking of using :false as the special value that will set the equivalent TOML value to boolean false.\nTOML does not define a null value, but if the Lisp value is nil, that key will simply not be translated to TOML.\nMapping lists to TOML\u0026nbsp;# Mapping lists was simple.. because in Lisp, a list value of course looks like \u0026#39;((foo . (1 2 3 4 5))) 😃.\nTable 2: Mapping of list Lisp data to TOML Lisp S-exp TOML '((list_key1 . (1 2 3))) list_key1 = [1, 2, 3] '((list_key2 . (\u0026quot;a\u0026quot; \u0026quot;b\u0026quot; \u0026quot;c\u0026quot;))) list_key2 = [\u0026quot;a\u0026quot;, \u0026quot;b\u0026quot;, \u0026quot;c\u0026quot;] Mapping lists of lists to TOML\u0026nbsp;# Table 3: Mapping of list of list Lisp data to TOML Lisp S-exp TOML '((lol_key . [(1 2) (3 4)])) lol_key = [ [1, 2], [3, 4] ] I was going to use \u0026#39;((lol_key . ((1 2) (3 4)))) as the reference Lisp expression for lol_key = [ [1, 2], [3, 4] ]. But I found out that json-encode throws an error if you pass it that expression! I don\u0026rsquo;t understand the reason for that error, and so I have asked for help on the help-gnu-emacs mailing list.\nBut while that question gets resolved, I wanted to move forward with the spec definition. After some trial-and-error and reverse-engineering json.el I knew how I wanted TOML to look. So I used an online JSON/TOML converter to convert that TOML snippet to JSON, and then used json-read to convert JSON to Lisp expression. , I learned that list of list data needs to be represented using a Vector type, and so \u0026#39;((lol_key . [(1 2) (3 4)])) would be the correct expression \u0026ndash; Notice the use of square brackets instead of parentheses for the outer vector.\nMapping other object types\u0026nbsp;# Once I figured out how to map the above data types, mapping Lisp data to TOML Tables aka dictionaries and arrays of Tables was a breeze Of course, the breeze is referring to the ease of writing the spec for these 😆. Implementation-wise, the tables, arrays of tables, and the especially nested variants of those are going to be the most challenging. .\nClosing\u0026nbsp;# It was fun coming up with an initial draft of the specification for this library. My next steps would be to gradually add TOML generator functions (as I find time) to this library, along with ERT tests! Eventually, I will remove the existing TOML generation code from ox-hugo and depend on this library.\nGetting back to my plan for exporting :LOGBOOK: drawers in ox-hugo, based on this spec, I will need to construct this date in Emacs-Lisp:\n(org_logbook . (((timestamp . 2022-04-08T14:53:00-04:00) (note . \u0026#34;This note addition prompt shows up on typing the `C-c C-z` binding.\\nSee [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).\u0026#34;)) ((timestamp . 2018-09-06T11:45:00-04:00) (note . \u0026#34;Another note **bold** _italics_.\u0026#34;)) ((timestamp . 2018-09-06T11:37:00-04:00) (note . \u0026#34;A note `mono`.\u0026#34;)))) Code Snippet 1: Example data from a :LOGBOOK: drawer in Lisp format .. will translate to this in TOML:\n[[org_logbook]] timestamp = 2022-04-08T14:53:00-04:00 note = \u0026#34;\u0026#34;\u0026#34;This note addition prompt shows up on typing the `C-c C-z` binding. See [org#Drawers](https://www.gnu.org/software/emacs/manual/html_mono/org.html#Drawers).\u0026#34;\u0026#34;\u0026#34; [[org_logbook]] timestamp = 2018-09-06T11:45:00-04:00 note = \u0026#34;\u0026#34;\u0026#34;Another note **bold** _italics_.\u0026#34;\u0026#34;\u0026#34; [[org_logbook]] timestamp = 2018-09-06T11:37:00-04:00 note = \u0026#34;\u0026#34;\u0026#34;A note `mono`.\u0026#34;\u0026#34;\u0026#34; Code Snippet 2: Same example data from the :LOGBOOK: drawer translated to TOML format Check out the below link where I have documented the equivalent expressions between Lisp, TOML and JSON for all the object types.\n👉 tom​el​r Specification\n"},"name":"Defining tomelr – A library for converting Lisp expressions to TOML","published":"2022-04-28T00:08:00-04:00","summary":"Creating a specification for an Emacs-Lisp library to convert Lisp\ndata expressions into easy-to-read TOML strings.","type":"entry","url":"https://scripter.co/defining-tomelr/"}],"name":"toml","type":"feed","url":"https://scripter.co/tags/toml/"}