This is ekg.info, produced by makeinfo version 7.1.1 from ekg.texi.

INFO-DIR-SECTION Emacs
START-INFO-DIR-ENTRY
* EKG: (ekg).           A note taking application for emacs using sqlite.
END-INFO-DIR-ENTRY


File: ekg.info,  Node: Top,  Next: Introduction,  Up: (dir)

ekg, the Emacs Knowledge Graph
******************************

author: Andrew Hyatt

   This is the ekg manual, which describes the operation and
customization of the ekg package.  All information here is relevant to
the released version only.

   The README is also informative, has screenshots, and can be found in
the source and in the git repository, which is at
<https://github.com/ahyatt/ekg>.

* Menu:

* Introduction::
* Installation::
* Changelog::
* Database::
* Concepts and data model in ekg::
* Understanding and editing the note buffer::
* Capturing notes::
* Viewing tags or notes::
* Searching for notes::
* Magic tags::
* The trash::
* Links to ekg in org-mode::
* Importing from org-roam::
* Backups::
* Database maintenance::
* Customizing ekg::
* Integration with ekg::
* Extras::
* Design::

-- The Detailed Node Listing --

Installation

* Sqlite dependency::
* Installing via Melpa::
* Installing by hand::

Changelog

* Version 0.9.2: Version 092.
* Version 0.9.1: Version 091.
* Version 0.9.0: Version 090.
* Version 0.8.0: Version 080.
* Version 0.7.1: Version 071.
* Version 0.7.0: Version 070.
* Version 0.6.4: Version 064.
* Version 0.6.3: Version 063.
* Version 0.6.2: Version 062.
* Version 0.6.1: Version 061.
* Version 0.6.0: Version 060.
* Version 0.5.1: Version 051.
* Version 0.5: Version 05.
* Version 0.4.3: Version 043.
* Version 0.4.2: Version 042.
* Version 0.4.1: Version 041.
* Version 0.4: Version 04.
* Version 0.3.3: Version 033.
* Version 0.3.2: Version 032.
* Version 0.3.1: Version 031.
* Version 0.3: Version 03.
* Version 0.2.1: Version 021.
* Version 0.2: Version 02.

Understanding and editing the note buffer

* Drafts::
* A warning about org-mode::

Capturing notes

* Templates::
* Changing the initial tags of a note::
* Inline commands::

Viewing tags or notes

* Commands in the notes buffer::
* Customizing note display in ekg-notes-mode::

Customizing ekg

* Hidden Tags::
* Text Truncation Method::

Extras

* Embeddings::
* Logseq::
* LLM::
* Auto save::
* Denote::
* Apple Notes::
* Email::
* Org Tasks::
* Agent::

Logseq

* Exporting::
* Importing::

LLM

* Augmenting notes with LLM output::
* Using ekg notes as prompts::
* Querying your ekg database::

Apple Notes

* Setup::
* Syncing::
* Tags::
* Content conversion::
* Limitations::

Org Tasks

* Data model::
* Org agenda integration::
* Capturing tasks::
* Changing task state::
* The ekg-org-view::
* Agent integration::
* Command-line tools::

Agent

* Optional connected-data tool libraries::
* Agent log mode continuing and cancelling::
* AGENTS.md support: AGENTSmd support.
* Command-line tools for agent integration::

Design

* The triple database::
* The metadata section::



File: ekg.info,  Node: Introduction,  Next: Installation,  Prev: Top,  Up: Top

1 Introduction
**************

The ekg module is a simple but opinionated note taking application, for
emacs.  It is a substitute for such other emacs applications such as
org-roam or denote.  ekg stands for _emacs knowledge graph_.

   Data is completely stored in a sqlite database.  Notes are organized
around tags, and you can view many notes by looking at one or more tags.
This provides a read-only view of various notes, which you can navigate
between and choose to edit in a separate buffer.

   The editing of notes combines the editing of the text of the note
with various properties stored in the database such as the tags of the
note.


File: ekg.info,  Node: Installation,  Next: Changelog,  Prev: Introduction,  Up: Top

2 Installation
**************

* Menu:

* Sqlite dependency::
* Installing via Melpa::
* Installing by hand::


File: ekg.info,  Node: Sqlite dependency,  Next: Installing via Melpa,  Up: Installation

2.1 Sqlite dependency
=====================

If using Emacs version 29, you likely have ‘sqlite’ as a library
compiled into Emacs.  Prior versions use ‘emacsql’ for access to sqlite.
If you are using a pre-Emacs 29 version, you need to have ‘sqlite’
installed on your system.


File: ekg.info,  Node: Installing via Melpa,  Next: Installing by hand,  Prev: Sqlite dependency,  Up: Installation

2.2 Installing via Melpa
========================

This is the easiest and recommended way to install ekg.  If you have not
set up Melpa yet, follow the instructions at
<https://melpa.org/#/getting-started>.

   There is no need to use Melpa Stable, because development happens in
a branch and is not integrated until it is deemed stable.  When bugs are
discovered, fixes are pushed to the main branch soon, so it is usually a
good idea to keep your version up to date.

   There are many ways to use Melpa to download ekg, but using
‘use-package’ is the easiest way, and recommended because it allows you
to get the package and configure it in one place.  An example use is
below.

     (use-package ekg
       :bind (([f11] . ekg-capture)))


File: ekg.info,  Node: Installing by hand,  Prev: Installing via Melpa,  Up: Installation

2.3 Installing by hand
======================

If you wish to install by hand, you need to make sure to install the
triples library, found in GNU ELPA, and at
<https://github.com/ahyatt/triples>.

   Clone the ekg library, from whatever branch you would like to use
(‘main’ corresponds to the release version, and ‘develop’ is where
development of the next version happens).  Add your source directory and
require ekg.  The following is an example assuming you cloned ekg into
‘~/src/ekg’, and the triples library is already installed.

     (add-to-list 'load-path "~/src/ekg")
     (require 'ekg)


File: ekg.info,  Node: Changelog,  Next: Database,  Prev: Installation,  Up: Top

3 Changelog
***********

* Menu:

* Version 0.9.2: Version 092.
* Version 0.9.1: Version 091.
* Version 0.9.0: Version 090.
* Version 0.8.0: Version 080.
* Version 0.7.1: Version 071.
* Version 0.7.0: Version 070.
* Version 0.6.4: Version 064.
* Version 0.6.3: Version 063.
* Version 0.6.2: Version 062.
* Version 0.6.1: Version 061.
* Version 0.6.0: Version 060.
* Version 0.5.1: Version 051.
* Version 0.5: Version 05.
* Version 0.4.3: Version 043.
* Version 0.4.2: Version 042.
* Version 0.4.1: Version 041.
* Version 0.4: Version 04.
* Version 0.3.3: Version 033.
* Version 0.3.2: Version 032.
* Version 0.3.1: Version 031.
* Version 0.3: Version 03.
* Version 0.2.1: Version 021.
* Version 0.2: Version 02.


File: ekg.info,  Node: Version 092,  Next: Version 091,  Up: Changelog

3.1 Version 0.9.2
=================

   • Fix issue computing batch embeddings without ‘vecdb’.


File: ekg.info,  Node: Version 091,  Next: Version 090,  Prev: Version 092,  Up: Changelog

3.2 Version 0.9.1
=================

   • Improve ‘ekg-agent’ quality and token use
   • Allow the user to change the provider for agent commands
   • Fix warnings when displaying images
   • Improve reliability and fix issues in Apple Notes sync
   • Only perform database upgrades when the code changes


File: ekg.info,  Node: Version 090,  Next: Version 080,  Prev: Version 091,  Up: Changelog

3.3 Version 0.9.0
=================

   • Added the ability to sync with Apple Notes.
   • Changed how note metadata is represented in the note editing UI.
   • Added ‘ekg-agent’, an autonomous agent that can help you with your
     notes, and perform other tasks using notes as memory.
   • Added ‘ekg-org’, an org-mode task integration module.  Tasks are
     stored in the ekg database and appear in the org agenda via a
     virtual file.  Includes ‘ekg-org-view’, an interactive hierarchical
     task view for collaborating with AI agents on task management.
   • Pushed more configuration into the triple database itself.


File: ekg.info,  Node: Version 080,  Next: Version 071,  Prev: Version 090,  Up: Changelog

3.4 Version 0.8.0
=================

   • Add support for the ‘vecdb’ package, for embedding storing and
     retrieval.
   • Add ‘ekg-truncation-method’ variable to control how truncation
     happens.
   • Fix various truncation issues with quotes, underscores and all
     sorts of other symbols.


File: ekg.info,  Node: Version 071,  Next: Version 070,  Prev: Version 080,  Up: Changelog

3.5 Version 0.7.1
=================

   • Don't show inactive notes with ‘ekg-search’.


File: ekg.info,  Node: Version 070,  Next: Version 064,  Prev: Version 071,  Up: Changelog

3.6 Version 0.7.0
=================

   • Add incremental search with ‘ekg-search’.
   • Add formatting options for ekg note templates.
   • Added ‘ekg-show-notes-for-query’ using ‘triples-fts’.
   • Speed up for LLM calls with notes as context
   • Support multiple values in ‘ekg-llm-providers’ so that they can be
     chosen from.
   • Ensure SQL connection and schema for embedding calls.


File: ekg.info,  Node: Version 064,  Next: Version 063,  Prev: Version 070,  Up: Changelog

3.7 Version 0.6.4
=================

   • Fix display of extended structured data in notes.
   • Add contrib directory and ‘ekg-email’ as a contribution to import
     emails to notes in a structured way.
   • Fix inlines not expanding in the contextual notes in ‘ekg-llm’.
   • Set ‘default-directory’ to db location in notes buffers (by S
     Pearman (https://github.com/monkpearman)).
   • Add tags when saving and exiting notes for any linkified markdown
     links.
   • Generate embeddings in batch in ‘ekg-embedding-generate-all’.


File: ekg.info,  Node: Version 063,  Next: Version 062,  Prev: Version 064,  Up: Changelog

3.8 Version 0.6.3
=================

   • Fix issue in trying to get a single-valued item to show up in the
     metadata line.
   • Remove the never-used named, email, and person schema.
   • Fix issue with tag completions triggering incorrectly if they
     contained spaces.
   • Fix error when displaying unknown schema.
   • Improve JSONification of notes when sending context to LLMs.


File: ekg.info,  Node: Version 062,  Next: Version 061,  Prev: Version 063,  Up: Changelog

3.9 Version 0.6.2
=================

   • Add export to denote (https://github.com/protesilaos/denote) (by
     Jay Rajput (https://github.com/jayrajput))


File: ekg.info,  Node: Version 061,  Next: Version 060,  Prev: Version 062,  Up: Changelog

3.10 Version 0.6.1
==================

   • Run tag hooks when a valid tag is typed, instead of relying on
     completion, which often doesn't work.
   • Fix issue in ‘markdown-mode’ where the beginning of the tag values
     was incorrectly read-only.
   • Allow emojis as tags.


File: ekg.info,  Node: Version 060,  Next: Version 051,  Prev: Version 061,  Up: Changelog

3.11 Version 0.6.0
==================

   • Add tag and similar notes to the context when having LLMs add to or
     replace notes.  ‘ekg-llm-default-instructions’ replaces the
     previous variable, ‘ekg-llm-default-prompt’ (because the prompt now
     contains the instructions).
   • Add inline links for Markdown, as wiki links.


File: ekg.info,  Node: Version 051,  Next: Version 05,  Prev: Version 060,  Up: Changelog

3.12 Version 0.5.1
==================

   • Improve regex for inline tag linkification so we linkify more valid
     inline tags.
   • Improve logseq import to import notes without org structure, and
     use filetags.
   • Fix inline tag bug that test and real failures.


File: ekg.info,  Node: Version 05,  Next: Version 043,  Prev: Version 051,  Up: Changelog

3.13 Version 0.5
================

   • Simplified trashed tags from using tag prefixes to using a simple
     trash tag (‘ekg-trash-tag’).
   • Added ‘ekg-show-notes-with-tag-prefix’.
   • Add tags while writing the body of notes by completing with "#".
     See *note Note text: Understanding and editing the note buffer.
     section for details.
   • Added functionality so that org-mode commands, and potentially
     others, narrow automatically so the metadata line and the read-only
     nature don't mess things up.  See *note Note text: Understanding
     and editing the note buffer. section for details.
   • ‘ekg-auto-save’, a new module by Qingshui Zheng
     (https://github.com/qingshuizheng).
   • Added canceling a note as an actual command, also by Qingshui Zheng
     (https://github.com/qingshuizheng).
   • Fix issue with trashing notes with tags that are partially trashed.
   • Saving while editing a note will no longer add the draft tag.  This
     means that if you are editing a draft, it will not keep it as a
     draft, so it will become a normal note.
   • Fix issue with using ekg-llm with vertex, which doesn't understand
     system prompts.
   • Fix metadata face which didn't work well with all themes - now the
     metadata section is just bolded.
   • Improve display regenerating embeddings via
     ‘ekg-embedding-generate-all’.
   • Change multi-title note display from using newlines to commas
   • Made deleting titles and resources possible, and properly skipped
     empty properties for multi-line properties.
   • Handle backup errors better, warning and proceeding if not a forced
     backup, erroring out with a better message if forced.
   • Don't show trashed notes in drafts.


File: ekg.info,  Node: Version 043,  Next: Version 042,  Prev: Version 05,  Up: Changelog

3.14 Version 0.4.3
==================

   • Added autoloads
   • Fixed issue that could occur when saving with malformed buffers.


File: ekg.info,  Node: Version 042,  Next: Version 041,  Prev: Version 043,  Up: Changelog

3.15 Version 0.4.2
==================

   • Switch LLM chat output to streaming when available.
   • Fix inclusion of title-based transclusion ">t", which included the
     "t" as part of the completion.
   • Fix tag renaming possibly causing duplication.
   • Ensure renamed tags are normalized.
   • Support metadata where fields are specified via multiple property
     lines and make "title" such a field, so now titles can have commas.


File: ekg.info,  Node: Version 041,  Next: Version 04,  Prev: Version 042,  Up: Changelog

3.16 Version 0.4.1
==================

   • Fix issues using default emacs in-buffer completion, and allowing
     completion in places we shouldn't.
   • Add ‘ekg-embedding-generate-on-save’ and
     ‘ekg-embedding-disable-generate-on-save’ to turn off generating
     embeddings for notes.


File: ekg.info,  Node: Version 04,  Next: Version 033,  Prev: Version 041,  Up: Changelog

3.17 Version 0.4
================

   • Added ability to save in-progress notes.
   • Added "magic tags", tags that cause elisp to be executed.  See the
     *note magic tags: Magic tags. section for more detail.
   • Added ‘ekg-llm’, a separate module, so LLMs can append to or
     rewrite notes, using other notes as prompts.  As with
     ekg-embeddings, a Open API key is required.
   • Added ‘ekg-capture-file’ to save notes associated with a file, or
     go that same note.
   • Depend on the llm package (https://github.com/ahyatt/llm) for
     embeddings and llm functionality, so the user can choose different
     providers.
   • Improved ‘ekg-clean-db’ to fix bugs and change empty-note deletion
     logic.
   • Improved ‘ekg-embedding’ package to make it more robust to missing
     embeddings (like what can happen if you save notes without loading
     the package first).
   • Added commas to the tag header.
   • Made separator in ‘ekg-capture-mode’ and ‘ekg-edit-mode’
     customizable.
   • Fixed display of org notes to have properly formatted links and
     images.  Links can be opened with ‘[C-c o]’.
   • Fixed bug where in embedding search and buffer similarity, the
     highest match was discarded.
   • Fixed bug in title transclusion for company users.
   • Changed template behavior to also use parent tags (so templates on
     "foo" tag, will work if the note tag is "foo/bar").

     Thanks to contributors Gleek (https://github.com/Gleek) and
     Qingshui Zheng (https://github.com/qingshuizheng).


File: ekg.info,  Node: Version 033,  Next: Version 032,  Prev: Version 04,  Up: Changelog

3.18 Version 0.3.3
==================

   • Fix native compilation errors.  Upgrade to triples version 0.3.5,
     which contains more important bug fixes.


File: ekg.info,  Node: Version 032,  Next: Version 031,  Prev: Version 033,  Up: Changelog

3.19 Version 0.3.2
==================

   • Upgrade to triples version 0.3.2, which contains important bug
     fixes.


File: ekg.info,  Node: Version 031,  Next: Version 03,  Prev: Version 032,  Up: Changelog

3.20 Version 0.3.1
==================

   • Upgrade to triples module 0.3, which changes how integers are
     stored in the built-in sqlite (for users of Emacs 29+).  Users of
     sqlite will have their database automatically upgraded.  A backup
     will always be made beforehand - you may want to find it (alongside
     your normal emacs backups), and make sure to keep it around in case
     the upgrade went wrong in some way.  *Important: if you created
     your database before this version on the built-in sqlite, and
     afterwards switched to emacsql, you must switch back to the
     built-in sqlite for the upgrade.*
   • Store the ekg version in the database so we know when we need to do
     updates in the future.
   • Remove older database updates that should no longer be needed.
   • Ensure we always are connected to the database before any call to
     the database happens.
   • Make ekg-close interactive.
   • Fixes to ignore bad embeddings which otherwise would cause errors.
   • Added the ability to kill notes in a notes view, which does not
     change the database, it only alters the view.  Thanks to Jay Rajput
     for the contribution!
   • Fix for tag cleanups, which were sometimes not cleaned up if the
     tag had other data (such as embeddings).
   • Added variable ‘ekg-embedding-text-selector’ with a default
     function so that large notes can have their embeddings taken.
   • Added ‘ekg-get-notes-with-title’, which is offered as a useful
     function for clients.


File: ekg.info,  Node: Version 03,  Next: Version 021,  Prev: Version 031,  Up: Changelog

3.21 Version 0.3
================

   • Added inline commands, see the *note inlines: Inline commands.
     section for more detail.
   • Added customization of note display, using inline commands.
   • Added logseq import / export in its own module, and removed it from
     the ‘ekg-org-roam’ module.  See the *note logseq: Logseq. section
     for more details.
   • Improve window management, now we play nicer with customized window
     configuration, and now opening a list of notes will also switch to
     that window.
   • Improved metadata overlay look and function, it now looks just like
     ‘message-mode’, which hopefully will help with people's intuitions
     on how it works.
   • Added a blank line between notes in notes list buffers, for a
     cleaner look.
   • Renamed ‘ekg-rename-tag’ to ‘ekg-global-rename-tag’ to clarify this
     isn't for changing an individual tag in a note.
   • Added arg prefix behavior to ‘ekg-notes-delete’ to allow deleting
     notes without a warning.
   • Made ‘ekg-capture’ have unique buffer names, so the user can
     capture multiple notes at the same time.

   Thanks especially to users and contributors: Jay Rajput
(https://github.com/jayrajput), Qingshui Zheng
(https://github.com/qingshuizheng), And cuprum
(https://github.com/cuprum).


File: ekg.info,  Node: Version 021,  Next: Version 02,  Prev: Version 03,  Up: Changelog

3.22 Version 0.2.1
==================

   • Removed ‘ekg-notes-remove’, which removed one or more tags from a
     note from a note list buffer.
   • New keybinding, "q" in the notes buffer, which kills the buffer
     (thanks to Jay Rajput for the idea).


File: ekg.info,  Node: Version 02,  Prev: Version 021,  Up: Changelog

3.23 Version 0.2
================

   • Added hooks ‘ekg-add-schema-hook’, ‘ekg-note-pre-save-hook’,
     ‘ekg-note-save-hook’, ‘ekg-note-delete-hook’, and
     ‘ekg-note-add-tag-hook’ to enable customization.
   • New commands ‘ekg-show-notes-latest-captured’,
     ‘ekg-show-notes-latest-modified’, for showing notes created or
     modified recently.
   • Introduced variable ‘ekg-notes-size’ to control the default page
     size for limited views such as ‘ekg-show-notes-latest-captured’.
   • Added templating.
   • Added embedding as an optional add-on, to enable note similarity
     and note search; requires an account at OpenAI or similar embedding
     provider.
   • Added new function ‘ekg-active-notes’ to easily get all non-trashed
     notes.
   • Improved ability to have note list buffers that have flexible
     titles and operation, notably improving the
     ‘ekg-show-notes-in-trash’ command.
   • Standardized buffer names for ‘ekg-notes-mode’ buffers, which all
     are prepended with "ekg" and surrounded by asterisks, to denote
     that they are non-file-based.
   • Added the documentation you are reading right now.
   • Fixed bug interfering with completion at the beginning of the tag
     property line.
   • Fix for ‘ekg-notes-refresh’ incorrectly calling ‘ekg--show-notes’.


File: ekg.info,  Node: Database,  Next: Concepts and data model in ekg,  Prev: Changelog,  Up: Top

4 Database
**********

By default, ekg uses the default triple database, which is set in the
variable ‘triples-default-database-filename’.  The default value of this
is ‘~/.emacs.d/triples.db’.  You can specify a different name if you
want the ekg database to not be shared with any other user of the triple
package, by customizing the variable ‘ekg-db-file’.  When this is ‘nil’,
it leaves the filename up to the triples package.


File: ekg.info,  Node: Concepts and data model in ekg,  Next: Understanding and editing the note buffer,  Prev: Database,  Up: Top

5 Concepts and data model in ekg
********************************

The ekg package is built on a flexible database scheme called "triples",
where everything is stored as a graph structure: a subject, a predicate,
and an object.  The implication for the ekg package is that new kinds of
data are easy to add, and live alongside other data.  Values of
properties, stored as "objects", can themselves have values by adding
data where the same value is the "subject".  If you plan to do extensive
integration work in elisp, it will help to understand these concepts,
and the best way to do so is reading the triples package README.

   For notes, we can think of the subject of the triples as an ID.
Notes are created, and have the following types by default, with the
type having properties.
   • ‘tagged’: Tags
   • ‘text’: Text, its major mode, and any inline commands.
   • ‘time-tracked’: Creation time and modification time
   • ‘titled’: Title

   The ID for notes is by default an integer UUID.  However, you can
have notes about anything.  In EKG an ID can be a resource identifier as
well, such as a URL.  When this happens, the ID is interesting data in
its own right.

   Tags may have spaces, but cannot have commas, which are used to
separate them when showing them to the user and parsing them back out
into properties to store.  Tags also may not contain uppercase letters.

   Because of the triples model, there is data about the tags for each
note.  Tags themselves just have type markers indicating they are tags,
and can dynamically query for all notes with their tag, so tags always
have a current list of notes with their tag.


File: ekg.info,  Node: Understanding and editing the note buffer,  Next: Capturing notes,  Prev: Concepts and data model in ekg,  Up: Top

6 Understanding and editing the note buffer
*******************************************

The note buffer is a buffer of the particular mode for the node, which
can change based off the setting, and can be different for different
notes.  It acts mostly just like that mode, but has a header that
displays some of the properties of the note, and lets you edit those
properties.

   The properties shown in the note property area come from the data
stored in the database for the entity.  At a minimum, there will be
tags.  Some properties are not useful to display (for example, embedding
vectors or internal parent/child IDs).  These can be hidden by adding
the property keyword to ‘ekg-header-hidden-properties’.  Packages such
as ‘ekg-embedding’ and ‘ekg-org’ automatically add their internal
properties to this list.

   Some properties need formatting before display.  For example, in the
‘ekg-org’ module, scheduled dates and deadlines are stored as epoch
seconds but should appear as human-readable dates.  The variable
‘ekg-property-format-functions’ is an alist mapping property keywords
(e.g.  ‘:org/scheduled’) to functions that take a single value and
return a display string.  Packages can register their own formatters by
adding entries to this alist; ‘ekg-org’ does this automatically for
‘:org/scheduled’ and ‘:org/deadline’.

   For the common cases, there are special commands, available under the
‘C-c /’ prefix keymap:

   • ‘a’: ‘ekg-note-add-tag’, to add a tag.
   • ‘r’: ‘ekg-note-remove-tag’, to remove a tag.
   • ‘t’: ‘ekg-note-add-title’, to add a title.
   • ‘T’: ‘ekg-note-remove-title’, to remove a title.
   • ‘c’: ‘ekg-note-change-title’, to change an existing title, or add
     one if none exist.
   • ‘p’: ‘ekg-note-edit-property’, to edit any property, including the
     "resource" of the note, which is its ID.  For editing, multi-values
     properties are editable as a list with values separated by ‘", "’.
     Maintain the space after the comma when editing.
   • ‘i’: ‘ekg-edit-add-inline’, to add an inline command.

     When altering properties, nothing is saved until the note is saved.

   The buffer has the text of the note.

   There are three modes for the note text: ‘text-mode’,
‘markdown-mode’, and ‘org-mode’.  More can be added by customizing the
variable ‘ekg-acceptable-modes’, just make sure its a mode that makes
sense for notes.  The default mode is configured in
‘ekg-capture-default-mode’, but can be changed when capturing with the
command ‘ekg-change-mode’.

   The note text provides various options for completions.  The most
common completion is typing tags inline while writing notes.  These tags
will be added to the note automatically upon saving, regardless of
whether completion is used or not.  The tag completion is triggered by
the "#" symbol.  In ‘org-mode’, if ‘ekg-linkify-inline-tags’ is set to
non-nil (which is the default), the tag will be turned into an org link
to the tag.  It is necessary to enclose the tag in square brackets to be
detected as an inline tag.  In ‘markdown-mode’, the tag will be turned
into a wiki link (denoted by double square brackets).  It is acceptable
to finish the completion with a tag that is not currently defined, as
the tag will be added when the note is saved.  The detection of
plaintext tags of various types can be turned off by setting
‘ekg-inline-populate-inline-text-tags’ to ‘nil’.  There are other
customizable tag symbols available, indicating different prefixes.  By
default, in addition to the "#" symbol (representing a tag with that
name), there is also the "@" symbol for tags prefixed with "person/",
and the "!"  symbol for tags prefixed with "idea/".  These other symbols
and the prefixes they mean are controlled by
‘ekg-inline-custom-tag-completion-symbols’.  By default, we have "@"
which will denote a tag with the "person/" prefix, and "!"  which will
denote a tag with the "idea/" prefix.  So if, in either ‘org-mode’ or
‘markdown-mode’, the text has the following:

     Everything related to #[emacs] should be colored #ffa500.  This is the opinion of @[rms].

   the tags that will be detected are "emacs" and, because of the
special "@" prefix that indicates a tag prefix, "person/rms".  The color
will not be picked up as a tag, because it is not enclosed in brackets.
This helps us avoid false positives.

   In ‘org-mode’ and ‘markdown-mode’, linkified links, which you get
with tag completion if ‘ekg-linkify-inline-tags’ is true, the links will
be detected as tags, and the tags will be added to the list of note tags
on saving and exiting the note.

   There is no functionality to remove inline tags that are deleted in
the tag section.  If an inline tag is not deleted in the note text, the
tag will be re-added.

   Another type of completion is for inline commands, the ">t"
completion, mentioned in the *note Inline commands: Inline commands.

   The metadata section above the notes often can be problematic for
some commands, especially org commands.  Because of this, before
commands are executed, we check the command name against the
‘ekg-command-regex-for-narrowing’, and if there is a match to one of the
regexes, we narrow to the note section just when the command is running.
It defaults to all org-insert commands and ‘org-meta-return’, but if
there are any weird behaviors caused by the metadata section, consider
customizing this variable.  Right now this just works for keybindings,
and not using ‘execute-extended-command’.

* Menu:

* Drafts::
* A warning about org-mode::


File: ekg.info,  Node: Drafts,  Next: A warning about org-mode,  Up: Understanding and editing the note buffer

6.1 Drafts
==========

Notes can be saved midway through editing, both for capturing and
editing notes.  The normal buffer-save keybinding (typically C-x C-s)
will save a draft.  A draft is like a normal note, but has a special
tag, by default "draft".  (This can be customized in ‘ekg-draft-tag’.)
Having this tag means the note doesn't show up in most views, much like
the notes in "trash".  Once a note is saved normally, it loses the draft
tag.

   See also the section on the *note auto save: Auto save., to see how
to turn on and set up auto save, which can automatically save drafts for
new notes.


File: ekg.info,  Node: A warning about org-mode,  Prev: Drafts,  Up: Understanding and editing the note buffer

6.2 A warning about org-mode
============================

Org-mode notes are primarily to use org-mode formatting on.  Org-mode
has a lot of functionality, but much of it depends on the assumption
that the buffer is all for use by org-mode (not true in this case,
because of the properties portion), and the assumption that the buffer
is visiting a file, which is also not true.  In particular, attachments
will not work.  However, the ‘ekg-org’ module (see *note Org Tasks: Org
Tasks.) provides org-agenda integration by presenting ekg tasks as a
virtual org file.


File: ekg.info,  Node: Capturing notes,  Next: Viewing tags or notes,  Prev: Understanding and editing the note buffer,  Up: Top

7 Capturing notes
*****************

‘ekg-capture’ is the command to capture a note.  In ekg this is probably
the most frequently used command.  It will create a new buffer called
‘*EKG Capture*’.  By default, it will have the current date tag, such as
"date/2023-02-21".

‘ekg-capture-url’ will capture a note associated with a URL resource,
and with a given title as the title of the page.  The idea is that the
note is annotating the reference itself as a "literature note".  The
title also appears as a tag, so other notes can reference this if
needed.  For example, if the URL is <http://example.com>, and the title
is "An example URL", the properties will be:

     Resource: <http://example.com> Tags: doc/an example url,
     date/2023-02-25 Title: An example URL

   Capturing URLs is a bit clunky as is.  If you can wrap it in a
function to supply the name and url of the active browser tab, then you
can create a much easier experience.  The following is an example for
users of Google Chrome on Mac OS X.

     (defun my/ekg-capture-url ()
       (interactive)

       (ekg-capture-url
        (do-applescript "tell application \"Google Chrome\" to return URL of active tab of front window")
        (do-applescript "tell application \"Google Chrome\" to return Title of active tab of front window")))

URL can also point to local files, which will be browsed using
‘find-file’ by default.  The idea is that you can tag files and folders
to make them easier to find.  Here is an example note similar to web
address URL:

     Resource:
     <file:///Users/andrewhyatt/notes/20230510T162600__emacs_init-file.org>
     Tags: doc/emacs config, date/2023-05-13, emacs/init Title: Emacs
     Config

   You can use the function ‘ekg-capture-file’ to either capture or edit
a note associated with a file from a buffer visiting that file.  (If
there is already a note associated with the buffer's file, the note will
be opened.)  You can use this to store TODOs and other notes about a
file.

   ekg opens web addresses in a browser using ‘browse-url’ and
everything else in Emacs using ‘find-file’.

A final way to capture notes comes from a buffer that is viewing a list
of notes, in ‘ekg-notes-mode’.  You can call ‘ekg-notes-create’, which
will capture a new note with whatever tags (if any) are associated with
the notes buffer.

To save any note that is being captured, press ‘C-c C-c’ or call
‘ekg-capture-finalize’.  To cancel, just kill the buffer.  You can also
abort with ‘C-c C-k’, or ‘ekg-capture-abort’, which will not only kill
the buffer but delete any draft saved.

* Menu:

* Templates::
* Changing the initial tags of a note::
* Inline commands::


File: ekg.info,  Node: Templates,  Next: Changing the initial tags of a note,  Up: Capturing notes

7.1 Templates
=============

Ekg comes with a built-in way to have templates.  When you add a tag to
a note, ekg searches for notes with both the tag being added, and the
tag "template".  Any note with those two tags will be added by default
to the text of the buffer.

   For example, if there is a note with tags "daily reflection" and
"template", with the text "What did you learn today?", adding the tag
"daily reflection" to a note will cause the text "What did you learn
today?"  to appear.  The parents of tags are also searched, so the same
thing will happen if the tag you add is "daily reflection/morning" - it
will get the template for "daily reflection" as long as it exists.

   The adding of templates happens whether initially when setting up the
capture buffer, or later when the user types a tag that is a valid tag.
Because of this, it's best to avoid adding templates to tags that are
prefixes of other tags you'd like to use, but don't want the template
on, because as soon as ekg sees the prefix that's a valid tag being
typed, it will trigger that tag's templates.

   You can choose a tag other than "template" as the trigger for this
templating behavior, by customizing ‘ekg-template-tag’.

   This functionality is enabled through the function
‘ekg-on-add-tag-insert-template’ in the variable
‘ekg-note-add-tag-hook’, and can be turned off by removing it from that
hook.

     (remove-hook 'ekg-note-add-tag-hook #'ekg-on-add-tag-insert-template)


File: ekg.info,  Node: Changing the initial tags of a note,  Next: Inline commands,  Prev: Templates,  Up: Capturing notes

7.2 Changing the initial tags of a note
=======================================

The variable ‘ekg-capture-auto-tag-funcs’ has a list of functions to
call to add tags.  Each function is called, and returns a list of tags
(or ‘nil’, the empty list), which are all added to a new note.  By
default, this variable has the function ‘ekg-date-tag’, which returns
the tag of today's date.  If you do not want this, you can remove this
function.  You can also add your own functions to add the year, the week
number, or any tag you feel is appropriate.


File: ekg.info,  Node: Inline commands,  Prev: Changing the initial tags of a note,  Up: Capturing notes

7.3 Inline commands
===================

An inline command is a way to insert generated content into notes.  A
command has a representation, and can be evaluated.  The representation
is an s-expression limited to a subset of functions.  An example of a
representation is "My .emacs file: %(transclude-file
\"~/.emacs.d/init.el\")".  When you are capturing or editing the note,
you can create this representation, or see one already created.  When
viewing the note in a notes buffer, the inline command is evaluated and
the results are inserted into the note.

   There are two kinds of inline commands: a normal command, and a note
command.  A normal command can do anything, and takes the form
"%(<command> <arg 1> <arg 2> ... <arg n>)".  In other words, this is
just like an elisp function call, except with a "%" in front.  When
executing one of these we look for a function starting with
‘ekg-inline-command-’.  So, for example, we have the following commands
available for use:

   • ‘%(transclude-note id <numwords>)’: Include the contents of another
     note.  ‘numwords’ is optional, and controls the maximum number of
     words to include.  If not included, there is no limit.
   • ‘%(transclude-file filename <numwords>)’: Include the contents of a
     file.  ‘numwords’ functions the same here as in ‘transclude-note’.
   • ‘%(transclude-website url <numwords>)’: Include the contents of a
     website.  As of now, no attempt is made to only include the "main
     content", so this is best suited to simple text sites that have
     content without any navigational elements.

   These are defined in ‘ekg-inline-command-transclude-note’, and so on.
A user can define new commands just by creating new functions that fit
this pattern.  All of these will be executed and content calculated
every time the note containing them is re-displayed.  Note that there is
currently no automatic refresh when the content being transcluded
changes.

   The other kind of inline command is a note command.  These function
similarly to normal inline commands, with the key difference that the
form is now "%n(<command> <arg 1> <arg 2> ... <arg n>)"; note the "n" in
front.  The difference here is that there is an implicit first argument
that is the note that is being displayed in the current context.  After
that note argument "<arg 1>" and so on will be added.  These are used
primarily for controlling the read-only display of notes in notes lists.
The note commands are primarily driven by types, with the idea that a
note can have many types, and each type has a note command that displays
information related to that type.  Note commands are defined in
functions with the prefix ‘ekg-display-note-’.  The following note
commands exist:

   • ‘%n(id <force>)’: Shows the ID of the note, if it is interesting.
     Interesting mainly means it isn't a random-seeming ID that we
     normally generate for notes, and is instead some sort of resource.
     If ‘force’ is true, then show it whether it is interesting or not.
   • ‘%n(text <numwords> <format1> <format2> ... <formatn>)’: The text
     of a note (with any inline commands calculated and their results
     displayed inline).  ‘numwords’ functions as noted above.  The
     ‘format’ options are an optional set of formatting symbols which
     control how the text is formatted.  See below for details.
   • ‘%n(tagged <numwords> <format1> <format2> ... <formatn>)’: The tags
     of a note.
   • ‘%n(time-tracked <format-str> <format1> <format2> ... <formatn>)’:
     The created and modified time of a note.  ‘format-str’, if passed,
     controls how the times are formatted (see documentation for
     ‘format-time-string’, default is ‘%Y-%m-%d’).
   • ‘%n(titled <numwords> <format1> <format2> ... <formatn>)’: The
     title of a note.
   • ‘%n(other)’: A special note command that will substitute itself
     with all type-relevant note commands that haven't already appeared.
     So, for example, if there is a type such as ‘person’, and a note
     has information with this type, that information will be shown in
     the ‘other’ command, as if it was substituted by ‘%n(person)’.
     However, if ‘%n(person)’ already appears as a command, it will not
     add it again in the ‘other’ command.

   The formatting options can be left out, as can ‘numwords’, for the
default implementation.  However, if you want to control the formatting,
there is a set of formatting options that exist.  That set is:
   • ‘oneline’, the results must be on one line only.  To see an example
     of this in use, see ‘ekg-oneliner-note-template’.
   • ‘plaintext’, the result must not have any string properties.

   The ‘%n(id <force>)’ is implemented in ‘ekg-display-note-id’,
‘%n(text <numwords>)’ is implemented in ‘ekg-display-note-text’, and so
on.  All these are designed to be useful for customizing the note
display (see *note Customizing note display in ekg-notes-mode:
Customizing note display in ekg-notes-mode.).  Because we want to have
these possibly not insert anything, each function must end with a
newline if the content is likely to be needing a line to itself.  The
functions must always return a string.  Although the default note
commands are all based around types, a note command could be anything
that needs a note.

   Inlines can be added by simply typing them, or a few special
commands.  ‘ekg-edit-add-inline’ will add an inline note or file.  For
notes, it will prompt to select a note by title or tag and then text.
For files, it will prompt for the file name.  The other way is to use
completion at point, by typing ">t" and completing by notes with titles.
After completion, the ">t" will be replaced with the correct
‘transclude-note’ command that refers to the titled note selected.  This
is only useful for notes with titles, since they are more easily
selected by completion.


File: ekg.info,  Node: Viewing tags or notes,  Next: Searching for notes,  Prev: Capturing notes,  Up: Top

8 Viewing tags or notes
***********************

There are several functions to view notes in various ways.  All of these
show a list of notes in read-only view, that can be navigated and
interacted with.  This is a ‘ekg-notes-mode’ buffer.

   ‘ekg-show-notes-with-tag’ will show all notes tagged with the given
tag.

   ‘ekg-show-notes-with-any-tags’ will show all notes that have any of
the tags given.

   ‘ekg-show-notes-with-all-tags’ will show all notes that have all of
the tags given.

   ‘ekg-show-notes-for-query’ will do a ranked text search, showing all
results.  The query is token-based so querying for individual characters
and syntax will not work; the query has to be words.  Prefixes such as
"tag", "title" and "text" can be used to restrict matches to tags,
titles and text respectively.  For example, the query "tag:apartment
sink" will show any article with "sink" in it (it could be anywhere,
even a tag), but it also has to have a tag with "apartment" in it.
Every term is ANDed together.  Use quotes if you want to quote a phrase
such as "tag:\"apartment problems\" sink".  This does not search
anything included via inline commands.  This is only possible for those
using the built-in sqlite backend in Emacs 29.1 or later.

   ‘ekg-show-notes-for-today’ will show the notes taken today.

   ‘ekg-show-notes-latest-captured’ will show a number of notes from
newest to oldest.  The number is 20 by default, but can be changed by
customizing ‘ekg-notes-size’.

   ‘ekg-show-notes-latest-modified’ will show a number of notes from
newest to oldest, but by modification time, not by creation time.  The
number is also 20 by default and can be changed by customizing
‘ekg-notes-size’.

   ‘ekg-show-notes-in-trash’ will show the notes in the trash (see the
*note trash: The trash. section for details on how this works).

* Menu:

* Commands in the notes buffer::
* Customizing note display in ekg-notes-mode::


File: ekg.info,  Node: Commands in the notes buffer,  Next: Customizing note display in ekg-notes-mode,  Up: Viewing tags or notes

8.1 Commands in the notes buffer
================================

The notes buffer is navigated via the following commands (the default
binding is also given):

   ‘ekg-notes-tag’ (‘t’), open another notes buffer showing notes with
one of the tags of current note.

   ‘ekg-notes-open’ (‘o’), edit the currently selected note.

   ‘ekg-notes-delete’ (‘d’), trash the current note (or, if this is the
trash list, truly delete it).

   ‘ekg-notes-browse’ (‘b’), open the resource attached to the current
note, if it exists, otherwise do nothing.

   ‘ekg-notes-select-and-browse-url’ (‘B’), select from all the titles
of URL resources in the any of the notes, and browse the URL.

   ‘ekg-notes-refresh’ (‘g’), refresh the list of notes in the current
buffer, to make sure any new notes or removed notes are updated in the
list.

   ‘ekg-notes-create’ (‘c’), capture a new note with all the tags
associated with the list.

   ‘ekg-notes-next’ (‘n’), move selection to the next note.

   ‘ekg-notes-previous’ (‘p’), move selection to the previous node.

   ‘ekg-notes-any-note-tags’ (‘a’), open a new notes list showing any of
the tags that appear in the selected note.

   ‘ekg-notes-any-tags’ (‘A’), open a new notes list showing any of the
tags that appear in any of the notes in the note list.  For example, if
the buffer was displaying notes with tag ‘emacs’, and there are two
notes displayed, one with tags ‘emacs’ and ‘org-mode’, and the other
with ‘emacs’ and ‘ekg’, a new buffer displaying notes with any of the
tags ‘emacs’, ‘org-mode’, or ‘ekg’ is created.

   ‘ekg-notes-kill’ (‘k’), kill a note from the current view.  This only
removes the note from the current buffer; the database is not changed.
If the view is refreshed, the note will come back.

   ‘q’ will kill the notes buffer.

   Many of these commands use the notion that notes lists have
associated lists of tags.  That is the case for many commands, but not
all.  For example, ‘ekg-show-notes-latest-captured’,
‘ekg-show-notes-latest-modified’, and ‘ekg-show-notes-in-trash’ have no
associated tags.


File: ekg.info,  Node: Customizing note display in ekg-notes-mode,  Prev: Commands in the notes buffer,  Up: Viewing tags or notes

8.2 Customizing note display in ‘ekg-notes-mode’
================================================

The main way to customize displays is via the variable
‘ekg-display-note-template’, which is a string that has inline commands
in it (normally inline note commands).  See the *note inlines: Inline
commands. section for more details on these commands.  Through changing
this, the ordering or inclusion of various type-related information can
be configured, or extra text added, or anything, really.

   The variable ‘ekg-format-funcs’ has functions to run to format what
ekg displays to the user.  Each format function runs in turn on a
temporary buffer with the note text in it, and can make whatever changes
it needs to before the buffer's contents are displayed in a note list.


File: ekg.info,  Node: Searching for notes,  Next: Magic tags,  Prev: Viewing tags or notes,  Up: Top

9 Searching for notes
*********************

There are a few different ways to search for notes, each with their own
advantages and disadvantages.

   ‘ekg-search’ will perform an incrementally displaying list of notes
where the text or title matches the query.  It is intended to be used as
a quick way to find text.  After the notes have been narrow down to the
extent you want, you can press the enter key to make a notes buffer out
of the remaining items.  There is no ranking or any semantic
sophistication here, it's a straight text match.

   ‘ekg-show-notes-for-query’ will display a notes buffer with entries
matching the query, in order of relevance.  This searches for text and
titles.  It has some sophistication due to the use of SQLite's FTS5
extension (https://www.sqlite.org/fts5.html).  This should work
essentially like a Google query, except it does not accept ‘AND’, ‘OR’,
or any other search operators.  All terms are treated as required.

   In the optional ‘ekg-embedding’ module, there is a third option:
‘ekg-embedding-search’, which displays a notes buffer similar to
‘ekg-show-notes-for-query’, but with everything ranked by semantic
relevance according to their embeddings.  See the *note embeddings:
Embeddings. section for more details.


File: ekg.info,  Node: Magic tags,  Next: The trash,  Prev: Searching for notes,  Up: Top

10 Magic tags
*************

Sometimes you want to have behaviors that are associated with particular
tags.  For example, if some of your notes are in Chinese, you may want
to tag them all with the same tag.  Going further, it would be nice if
all notes tagged with "chinese" had your favorite Chinese input method
on by default.  With magic tags, you can enable this tag-based
customization.

   This works in a similar manner to *note templates: Templates., except
that a template tag only takes effect when you add it, while a magic tag
takes effect both when first adding it and when editing a note with the
tag.  But they also share the same shortcoming: if the tag is a prefix,
it will trigger as soon as typed, even if you wanted to use a different
tag that is prefixed with the tag.

   Creating magic tags is also like creating templates.  You create a
note and use a special tag that indicates this tag is a magic tag.  That
special tag is "tag-defun" (but the name can be changed by customizing
the variable ‘ekg-function-tag’).  This tag is itself a "magic tag", and
once you add it to a note, the note will change to be in
‘emacs-lisp-mode’.  Notes co-tagged with this will take effect for any
notes with those co-tags (again, just like templates).  For this reason,
it's probably best to avoid having any date tags co-tagged, since users
probably don't want them to be magic tags.  To illustrate the example
that in this section, you could have a note with tags "chinese" and
"tag-defun".  This note could have the following content:

     (set-input-method 'chinese-b5-quick)

In this example, once a note is added with "chinese", this function will
be run, and all subsequent editing of the note will have this function
run.  Note that there can be only one elisp expression in the note; if
you have multiple, only the first will be used.  It is not advised to
have complicated elisp here, since it is not amenable to debugging.  The
code is run in the context of the note buffer, after the text has been
inserted.

   For tags that are a hierarchy, each level in the hierarchy is tried
in order, from least specific to most specific.  So, for example, if the
tag was "chinese/writing practice", first we would try "chinese", apply
any functions found there, then try "chinese/writing practice", and
apply any functions found there.


File: ekg.info,  Node: The trash,  Next: Links to ekg in org-mode,  Prev: Magic tags,  Up: Top

11 The trash
************

Notes deleted from note lists (‘ekg-notes-mode’) buffers are not deleted
outright, but rather put in the trash, which is done by adding the
‘ekg-trash-tag’, by default, "trash", to the note.  Any note with the
"trash" tag will not be shown in normal tag buffers.

   Trashed notes can be seen by calling ‘ekg-show-notes-in-trash’.  If
notes are deleted from this list via ‘ekg-notes-delete’ again, they are
deleted permanently.  The function ‘ekg-notes-delete’ will always delete
a note if the note is in the trash, and trash it otherwise.  If you want
to un-trash the note, you can remove the trash tag.


File: ekg.info,  Node: Links to ekg in org-mode,  Next: Importing from org-roam,  Prev: The trash,  Up: Top

12 Links to ekg in org-mode
***************************

Both notes in ekg and certain note list buffers can be stored and linked
to in org-mode.  To store a link to a note, you have to edit that note
and call ‘org-store-link’.  That function can also be called in a
‘ekg-notes-mode’ buffer created by ‘ekg-show-notes-with-any-tags’.
Other list types currently will just store their tags assuming the user
wants a link to a list with any of the tags in the list.


File: ekg.info,  Node: Importing from org-roam,  Next: Backups,  Prev: Links to ekg in org-mode,  Up: Top

13 Importing from org-roam
**************************

You can import your notes from org-roam.  This will turn all titles into
tags, and all links will become tags as well, as well as any tags
org-roam thought were in the document.  At the moment, the import is
started via executing elisp, since importing can be fairly
idiosyncratic, and ekg and org-roam have different ways of expressing
the same thing that you may want to change.  It's best if you looked
over ‘ekg-org-roam.el’ and see what is going on, but at least read the
following description before manually executing ‘(ekg-org-roam-import)’.

   The import is idempotent, so it always will import to the same
entities, overwriting older data with new data.  If you want to update
what is in ekg, you can just rerun the import.  In the import, titles
and links will be normalized to ekg's tag format (they will be downcased
and have any commas removed).  If you have tags you want to turn into
prefixes (which is a good idea for tags widely applied, which
essentially act as a categorization), you can add those tags to the list
at ‘ekg-org-roam-import-tag-to-prefix’.  For example,

     (setq ekg-org-roam-import-tag-to-prefix (append ekg-org-roam-import-tag-to-prefix '("idea" "person")))

   Then, when a note is found that is tagged with "idea", but with title
"emacs is a powerful tool", then the title in org-roam will be turned
into the ekg tag "idea/emacs is a powerful tool", and anything linked to
it will also get the same prefix.


File: ekg.info,  Node: Backups,  Next: Database maintenance,  Prev: Importing from org-roam,  Up: Top

14 Backups
**********

By default, the ekg package will back up its database, using the backup
functionality built into the triples library.  The backup behavior is
controlled by ‘ekg-default-num-backups’, set to ‘5’ by default, and
‘ekg-default-backups-strategy’, set to ‘daily’.  These are, on first use
of ekg, stored in the database itself, but it can be set again at any
time by running:

     (triples-backups-setup ekg-db ekg-default-num-backups ekg-default-backups-strategy)

   The strategy can be one of the defaults of ‘daily’, ‘weekly’,
‘every-change’, or ‘never’, and new methods can be defined as well.  See
the implementation in ‘triples-backups.el’ for more information.


File: ekg.info,  Node: Database maintenance,  Next: Customizing ekg,  Prev: Backups,  Up: Top

15 Database maintenance
***********************

You may occasionally notice that certain tags are obsolete and have no
notes, or notes exist that are empty, or various other annoyances.  You
can call ‘ekg-clean-db’, which will:

   • First, force a backup.
   • Remove all tags with no uses.
   • Remove notes with no text, or just a "*", which is something that
     often happens with org-mode buffers.

   Tags may need to be renamed because the concept has changed in some
way.  The command ‘ekg-global-rename-tag’ can quickly rename one tag to
another globally across the database, so all tags with the old tag now
have the new tag.  (Note that the new tag may already exist, in which
case this operation cannot be easily undone.)


File: ekg.info,  Node: Customizing ekg,  Next: Integration with ekg,  Prev: Database maintenance,  Up: Top

16 Customizing ekg
******************

You can customize the behavior of ekg in a number of ways.

   First, you can create your own schema to store your own data.  The
hook ‘ekg-add-schema-hook’ is called whenever the database is connected
to.  At that point, ekg adds all of its schema, and runs the hooks in
this variable.  Adding schema is idempotent, so it can be called any
number of times without causing problems.  Adding schema can be done by
calling the triples library.  For details on how to create schema, you
can either look at the ekg implementation for example, or the triples
library README for an overview of how it works.

   The ‘ekg-note-pre-save-hook’ is called before saving a note, and
‘ekg-note-save-hook’ is called after saving, but in the same database
transaction as the save.

   The ‘ekg-note-delete-hook’ is called when deleting a note.

   The ‘ekg-note-add-tag-hook’ is called when adding a tag, either via
the initial tags added to a new note, or tags added after completing a
new tag in the note's property list.

* Menu:

* Hidden Tags::
* Text Truncation Method::


File: ekg.info,  Node: Hidden Tags,  Next: Text Truncation Method,  Up: Customizing ekg

16.1 Hidden Tags
================

EKG allows modules or users to hide certain tags from normal view by
adding them to the ‘ekg-hidden-tags’ list.  Notes containing any of
these tags will not appear in standard note lists unless that specific
tag is explicitly searched for.

   By default, tags like ‘ekg-draft-tag’ and ‘ekg-trash-tag’ are already
treated as hidden.  Other modules, such as ‘ekg-agent’, use this list to
hide internal metadata notes.


File: ekg.info,  Node: Text Truncation Method,  Prev: Hidden Tags,  Up: Customizing ekg

16.2 Text Truncation Method
===========================

EKG provides a way to customize how text is truncated for display (e.g.,
in note lists or inline transclusions) and for generating embeddings.
This is controlled by the ‘ekg-truncation-method’ variable.

   You can set ‘ekg-truncation-method’ to one of the following values:
   • ‘'word’ (default): Text is truncated based on word counts.  This is
     suitable for languages that use spaces to separate words.
   • ‘'character’: Text is truncated based on character counts.  This
     method is recommended for Asian languages (such as Chinese,
     Japanese, Korean) that do not rely on spaces for word segmentation.

   When ‘ekg-truncation-method’ is set to ‘'character’:
   • Functions like ‘ekg-truncate-at’ will interpret their numerical
     argument (e.g., ‘numwords’ or ‘num’) as a character limit.
   • Numbers used in display templates (e.g., ‘%n(text 500)’) will refer
     to character limits.
   • The ‘ekg-embedding-text-selector-initial’ function (used by the
     ‘ekg-embedding’ module) will select text for embeddings based on a
     character limit.

   Users working with Asian languages, or those who prefer
character-based limits for other reasons, should set
‘ekg-truncation-method’ in their Emacs configuration:
     (setq ekg-truncation-method 'character)

   Be mindful that if you switch to ‘'character’ mode, numerical limits
you might have previously set for word counts (e.g., in
‘ekg-display-note-template’ or ‘ekg-note-inline-max-words’) will now be
interpreted as character counts.  This might require adjusting those
numbers to achieve your desired display length.  For example, a limit of
500 words is very different from 500 characters.


File: ekg.info,  Node: Integration with ekg,  Next: Extras,  Prev: Customizing ekg,  Up: Top

17 Integration with ekg
***********************

The ekg package is designed to be easy to integrate with.  For example,
if you want to create a note automatically in one of your functions, you
can write:

     (defun my/log-to-ekg (text)
       "Log TEXT as a note to EKG's date"
       (ekg-save-note (ekg-note-create :text text :mode 'text-mode :tags (list (ekg-tag-for-date) "log"))))

If you wanted to re-use an existing note and append to it, you can do
that as well.

     (defun my/log-to-ekg (text)
       "Log TEXT as a note to EKG's date and append if possible."
       (let ((notes (ekg-get-notes-with-tags (list (ekg-tag-for-date) "log"))))
         (if notes
             (progn
               (setf (ekg-note-text (car notes)) (concat (ekg-note-text (car notes)) "\n" text))
               (ekg-save-note (car notes)))
           (ekg-save-note (ekg-note-create :text text :mode 'text-mode :tags (list (ekg-tag-for-date) "log"))))))

   There isn't a special API, but the basic defuns such as
‘ekg-save-note’, ‘ekg-note-create-text’, ‘ekg-get-notes-with-tags’,
‘ekg-get-note-with-id’, along with the struct ‘ekg-note’ are good
starting points.  Capturing notes in different ways can be done by
wrapping ‘ekg-capture’, and is how functions such as ‘ekg-capture-url’
work.

If you add schema and you want the user to be able to modify it, you
should supply new alist entries to ‘ekg-metadata-parsers’ and
‘ekg-metadata-labels’.

Because inline commands exist, the complete text of a note should be
retrieved with ‘ekg-display-note’.  The function ‘ekg-note-text’, will
only get the text as stored, which is missing mode related text
properties and any text generated from inline commands.


File: ekg.info,  Node: Extras,  Next: Design,  Prev: Integration with ekg,  Up: Top

18 Extras
*********

The ekg module can have any number of functionality additions.  These
may appear as other packages with other maintainers, but some are
included as part of this package.

* Menu:

* Embeddings::
* Logseq::
* LLM::
* Auto save::
* Denote::
* Apple Notes::
* Email::
* Org Tasks::
* Agent::


File: ekg.info,  Node: Embeddings,  Next: Logseq,  Up: Extras

18.1 Embeddings
===============

The embeddings functionality, for integration with an LLM, can be turned
on by requiring the ekg-embeddings file and enabling it, such as:

     (require 'ekg-embedding)
     (ekg-embedding-generate-on-save)

   This module contains functionality to explore similar notes and
search using techniques associated with large language models.
Embeddings let you do searches at a semantic level, based on an
understood meaning that is separate from the words used.  For example,
if I have a note with a recipe for linguini, embeddings will let me see
that it is similar to notes about spaghetti, and not similar to notes
about cold fusion.  Because the search is not based on words, but
meaning derived from those words, notes that describe the same thing in
two different languages should be very similar.  In ekg these let you
find notes similar to a current note, or in fact any buffer.  You can
also do a query via embeddings.

   The idea behind an embedding is that it is an abstract representation
of text, represented as a multi-dimensional vector.  Because it is just
a vector, you can compare the distance between different embeddings, and
embedding vectors that are similar should represent similar concepts.
This can be used to find similar notes, but also to search, where the
search string is transformed into an embedding.

   The embedding interfaces with your preferred LLM provider via the
‘llm’ package.  This package allows the user to define their preferred
llm backends, which will be stored in ‘ekg-embedding-provider’.  Please
see the LLM module project page (https://github.com/ahyatt/llm) for a
complete description on how to do this, but an example would be the
following:

     (use-package ekg
       :init
       (require 'llm-openai)  ;; the specific LLM provider must be required
       (require 'ekg-embedding)
       (require 'ekg-llm)
       (ekg-embedding-generate-on-save)
       (let ((my-provider (make-llm-openai :key "my-openai-api-key")))
         (setq ekg-llm-provider my-provider
               ekg-embedding-provider my-provider)))

   The embedding provider must be the same for all notes.  If you want
to switch to a new provider, you will need to call
‘ekg-embedding-generate-all’ with a prefix argument (‘C-u M-x
ekg-embedding-generate-all’), which will regenerate all embeddings
asynchronously.  The embedding provider does not have to be the same as
the LLM provider (if you also use the *note LLM: LLM. add-on.)  Also
note that the provider will get the text of all your notes, so if that
bothers you, do not use any provider on a server.

   It is highly recommended to install the ‘vecdb’ package (installable
on GNU ELPA), to get access to fast embeddings from a variety of
possible backends.  An example of configuration to do that would be:

     (use-package vecdb
       :config
       ;; Config for qdrant, other providers for other backends are available, see
       ;; https://github.com/ahyatt/vecdb for details.  For this provider, you will
       ;; need a key and a URL.
       (defvar my-vecdb-provider (make-vecdb-qdrant-provider :api-key <yourkey> :url <url>))
       (setq ekg-vecdb-provider (cons ash/vecdb-provider (make-vecdb-collection :name "ekg" :vector-size 1536))))

   If ‘ekg-vecdb-provider’ is not set, the SQLite database is used,
which can get fairly slow when there is a lot of data.

   Once you have this set up, and you have already called ‘(require
'ekg-embedding)’ and ‘(ekg-embedding-generate-on-save)’ you can call
‘M-x ekg-embedding-generate-all’.  This may take a long time as each
embedding has to be generated separately with its own API call.  Once
you've done this, you can call, in ‘ekg-notes-mode’,
‘ekg-embedding-show-similar’ to get a list of similar notes.  You can
also call ‘ekg-embedding-search’ to perform a search over your notes
using embeddings.  In any buffer, you can call
‘ekg-embedding-show-similar-to-current-buffer’ to show notes similar to
whatever the text is in the current buffer.

   The variable ‘ekg-embedding-text-selector’ has a function that will
pre-process all text that is sent for embeddings.  The default value is
‘ekg-embedding-text-selector-initial’, which will estimate the size of
the tokens sent and limit the text to the first 8k tokens.  Right now
the function is tuned to the limits of Open AI's embedding framework,
and a different function may be needed for other embedding APIs.

   If you would like to stop generating embeddings for notes in a
session, you can call ‘(ekg-embedding-disable-generate-on-save)’.


File: ekg.info,  Node: Logseq,  Next: LLM,  Prev: Embeddings,  Up: Extras

18.2 Logseq
===========

ekg can sync with logseq, a PKMS application that can run on a laptop or
phone.  Logseq is particularly convenient as a way to view or enter
notes on your phone, and various synchronization solutions exist to sync
local files with your phone.  Because ekg and logseq have different
designs, these apps are not perfectly compatible.  The ekg and logseq
syncing is designed to favor ekg's system when a conflict arises.

   There are two ways to use logseq with ekg.  One is maintaining logseq
as an export-only copy of ekg data, where you don't plan to modify
anything in logseq, just using it to access your notes on other
platforms.  Exporting from ekg is destructive, though, so without an
initial import, *exporting will overwrite logseq files with data from
ekg, so it may destroy data*.  The other way is to sync bidirectionally.
This starts by importing anything from logseq that has never been
imported before, and then writing ekg's data on top.  This will preserve
data, but will lose the initial ordering of pages.  Both of these
methods, then, will significantly impact your logseq notes.  *It is
highly advised to back up your logseq files before starting*.

   To export to logseq, start by requiring the ‘ekg-logseq’ module and
setting up ‘ekg-logseq-dir’, which points to the base directory of your
logseq data (where there is a "pages" and "journals" directory):
     (require 'ekg-logseq)
     (setq ekg-logseq-dir "~/my/logseq")

   If you wish to maintain logseq as a read-only copy of ekg, just run
‘ekg-logseq-export’ when you wish to export data.  This currently may
take a few seconds to a minute, depending on how much data you have.  We
attempt to not write any files that are unchanged.  To have a
bidirectional synchronization, run ‘ekg-logseq-sync’, which will first
import data from logseq, then export data.

* Menu:

* Exporting::
* Importing::


File: ekg.info,  Node: Exporting,  Next: Importing,  Up: Logseq

18.2.1 Exporting
----------------

When exporting, it's important to understand the differences between ekg
and logseq.  Logseq is organized by pages, where one page is one file.
Within the page there are many sections, which can be individually
referenced.  The ekg integration treats logseq pages like ekg tags, and
logseq sections like ekg notes.  In logseq, the user mostly sees one
page at a time.  In ekg, notes are shown in a variety of contexts,
mostly tag related, but not always.  In logseq, a note lives in one page
and is referenced from other pages.  In ekg, each note has its own
identity and is tied to other notes solely via the tags it shares with
them.  To compensate for this difference, we export notes based on their
first non-date tag as the page where the text will appear, and reference
other tags, where they will appear as backlinks.  In addition, in
org-mode, notes in a page appear as top-level outlines, which are
supposed to have text for the outline node.  If there is an ekg note
with a title, the title will appear as the text, otherwise the outline
node will just read "Untitled note".  Because this initial headline is
where various properties are stored, and is followed immediately by
tags, it makes sense that this is a title instead of just part of the
content.

For example, take the following note:
     Tags: date/2023-04-05, ekg, logseq

     ekg can export into logseq!

This will be exported into "pages/ekg.org":

     #+title: ekg

     * Untitled note
     :PROPERTIES:
     :ID: 33134561605
     :EKG_HASH: 89471eadbd7cc56b088f5513c11f68cb1d11d045
     :END:
     #[[2023-04-05]] #[[logseq]]
     ekg can export into logseq

Each node points to its ID which is from ekg (but, if it was originally
imported, the ekg ID might originally be from logseq).  We also encode
the hash of the exported data.  This is to keep track of what was
exported, so we do not re-import it unless it has changed.  For now,
even if the data is changed, it is not re-imported.  Files for
"pages/logseq.org" and "journals/2023-04-05" will also be created,
although they won't have any content from this note.

When exporting, inline commands (see *note inlines: Inline commands.
section), are evaluated before exporting to logseq, with the exception
of note transclusions, which turn into logseq embeds to the same ID.
So, other kinds of transclusions or any other commands will evaluate to
whatever text they normally evaluate to when viewing the note before
exporting to logseq.  For example, if the note has a file tranclusion
inline command, the file contents will be exported to logseq.  Logseq
embeds are roughly equivalent to note transclusions, but only roughly,
since a key difference is that logseq embeds occupy their own lines and
appear visually distinct, and ekg transclusions don't.  Because of this,
some formatting strangeness between the two may happen.


File: ekg.info,  Node: Importing,  Prev: Exporting,  Up: Logseq

18.2.2 Importing
----------------

Imports from logseq will return all top-level items as separate notes.
So, for example, assuming we're reading from the logseq file
"pages/logseq.org":

     * This is my first time trying logseq  #testing
     * The org compatibility here is especially nice  #org

        It really helps me feel comfortable in using the various formatting options I had gotten used to.

This will turn into two notes, one that has text "* This is my first
time trying logseq #testing", and with tags ‘logseq’, and ‘testing’, and
the other with the rest of the text, with the tags ‘logseq’ and ‘org’.

There are a few things to be aware of.  In logseq, any level of the
hierarchy can have an id and be referenced separately.  In ekg, we don't
support notes inside of other notes, so these will be imported in the
context of the parent note, and won't be available to reference as its
own separate note.  Also, logseq has other functionality not supported
by ekg, such as queries and potentially anything provided by plugins.
These will be imported as-is to ekg, but without any corresponding
functionality.

Logseq embeds are imported as note transclusions.


File: ekg.info,  Node: LLM,  Next: Auto save,  Prev: Logseq,  Up: Extras

18.3 LLM
========

The ekg-llm module provides a second way to use large language models
(LLMs) with ekg, separately from the ekg-embeddings integration.  While
the ekg-embeddings module lets you find notes based on their meanings,
the ekg-llm module lets you prompt an LLM with the contents of a note,
and then capture the LLM's response in the note.

   As with ekg-embeddings, this is based on the ‘llm’ package, which
allows the user to define their preferred llm backends, which will be
stored in ‘ekg-llm-provider’.  Please see the LLM module project page
(https://github.com/ahyatt/llm) for a complete description of how to do
this, but an example would be the following:

     (use-package ekg
       :init
       (require 'llm-openai)  ;; the specific LLM provider must be required
       (require 'ekg-llm)
       (require 'ekg-embedding)
       (let ((my-provider (make-llm-openai :key "my-openai-api-key")))
         (setq ekg-llm-provider my-provider
               ekg-embedding-provider my-provider)))

   The embedding and LLM providers can be different.  The LLM provider
can change at will, while the embedding provider must be the same for
all embeddings in the database.  It is necessary to create both of these
providers, because some LLM functionality depends on having embeddings.

   The variable ‘ekg-llm-provider’, can be a single provider, similar to
‘ekg-embedding-provider’, or a list.  If it is a list, the first element
will be used by default.  However, when the user invokes
‘ekg-llm-send-and-append-note’ or ‘ekg-llm-send-and-replace-note’ with a
prefix argument (normally via ‘C-u M-x’), the user can select among the
different providers.

* Menu:

* Augmenting notes with LLM output::
* Using ekg notes as prompts::
* Querying your ekg database::


File: ekg.info,  Node: Augmenting notes with LLM output,  Next: Using ekg notes as prompts,  Up: LLM

18.3.1 Augmenting notes with LLM output
---------------------------------------

To send a note to an LLM and capture its response, call
‘ekg-llm-send-and-append-note’, which is by default bound to ‘[C-c .]’.
A prefix argument (‘[C-u C-c .]’) will let you edit the prompt,
temperature, and possibly the provider used before it is sent.  The
output from the LLM is appended at the end of the note, in a special
section.

   In addition to the contents of the note, ekg will construct a larger
prompt for the LLM.  The prompt consists of context about previous notes
that contain the tags of your note, and similar notes, which is what
will generate high-quality content that is appropriate in the context of
your notes.  It also contains instructions to the LLM to how to generate
the note text to be added or replaced.  The default instructions are a
fixed string you can configure in ‘ekg-llm-default-instructions’.  But
alternatively, you can create alternative prompts for different ekg tags
in the same way that *note templates: Templates. work, by creating a
note tagged with "prompt" and any other tag (the special "prompt" tag
can be changed by customizing ‘ekg-llm-prompt-tag’).  The alternate
prompt is created by appending all "prompt"-tagged notes.  Note that, as
with templates, hierarchical tags can have prompts attached at any or
all levels of the hierarchy.

   To take an example, imagine that you have a note tagged with ‘prompt’
and ‘recipe’, containing instructions saying the LLM should imagine
itself an authority on cooking and provide you helpful tips to improve
your recipes.  You then create a note with a child tag of ‘recipe’,
let's say ‘recipe/monkfish’, with some details of your attempt to cook
monkfish, and then hit ‘[C-c .]’.  Because ‘recipe’ is a parent of
‘recipe/monkfish’, ekg-llm will use these instructions instead of the
default one, and will also append your note, and place the LLM's
response in a special section at the end of your note.  For example:

     Making monkfish again.  It is thick but tends to be wet and hard to get a good
     sear on.  Maybe I should sous vide it and then blast it with the searzall torch?

     #+BEGIN_LLM_OUTPUT
     Monkfish can indeed be challenging to sear properly due to its high water
     content.  Sous-vide cooking followed by searing with a torch can be a great
     technique to achieve the desired result.  The sous-vide process will help to cook
     the fish evenly and retain moisture, while the searzall torch can give it a
     beautiful caramelized crust.  Just be sure to pat the fish dry before searing for
     better browning and use high heat to quickly sear the exterior without
     overcooking the inside.  Happy cooking!
     #+END_LLM_OUTPUT

   Instead of appending, the note can be replaced with the output of the
LLM by using ‘ekg-llm-send-and-replace-note’ which is bound to ‘[C-c
,]’.  As with the append command, using a prefix argument will let you
edit the instructions before sending it.

   All prompts sent from a note in org or markdown modes have a prelude
that notes the format of the input and expected output.  However, LLMs
typically will produce markdown regardless of what you ask it to do, so
if you want to use LLMs, you may want to use markdown as a default note
format.


File: ekg.info,  Node: Using ekg notes as prompts,  Next: Querying your ekg database,  Prev: Augmenting notes with LLM output,  Up: LLM

18.3.2 Using ekg notes as prompts
---------------------------------

ekg notes are especially well suited for LLM prompting, both because of
the ability to create prompts for different tags, and the ability to
transclude one note's contents within another note.  While each
"prompt"-tagged note should work as a standalone LLM prompt, it may be
helpful to build up a set of partial prompts that you can share among
many full prompts using transclusion.

   For example, imagine a prompt that is designed to give an
Aristotelian response to a note.  A note with tags "aristotle" and
"prompt" could have the basics: "You are Aristotle.  Give a response to
the note using Aristotle's writing style and ideas, referencing existing
works when possible."  But perhaps you also want some standard behaviors
found in other prompts, such as a prompt to encourage the LLM to ask you
questions when appropriate.  There may be many prompts in which that
sub-prompt may be applicable.  You can use transclusion *note inlines:
Inline commands. for this, adding the transclusion to the appropriate
part of the prompt.  You can then iterate on each sub-part, trying to
get the best behavior.

   Additionally, transclusion or other inline commands could help in
other ways in forming the prompt, by sharing your schedule, or your
current org agenda items as context to the LLM when it is necessary.
These advanced uses will require inline commands that are not part of
the base ekg package, but once written, they can be seamlessly used in
prompts.


File: ekg.info,  Node: Querying your ekg database,  Prev: Using ekg notes as prompts,  Up: LLM

18.3.3 Querying your ekg database
---------------------------------

If you also use embeddings, you can use the interactive function
‘ekg-llm-query-with-notes’ to find your notes that best match a query,
and send the LLM a prompt consisting of those notes.  This essentially
will let your notes act as a natural language queryable knowledge base.
It will work for queries in which you have the relevant information.
The answer to the query will appear in a new buffer.

   The initial part of the prompt instructing the LLM for this case is
defined in ‘ekg-llm-query-prompt-intro’.  This can be changed to tune
how the LLM responds to the query.

   Note that anything in your database could potentially be retrieved
and sent to the LLM, so if you have notes that you consider too private
to send for processing, you may not want to use this.


File: ekg.info,  Node: Auto save,  Next: Denote,  Prev: LLM,  Up: Extras

18.4 Auto save
==============

The ‘ekg-auto-save’ module is useful for users who enter longer notes,
so that the notes are protected against accidentally killing the buffer,
or emacs crashing, or any similar problem.  It is designed to work
similarly to the built-in auto-save functionality, and has it's own
variables that default to the auto-save equivalent.  So, for example,
there is ‘ekg-auto-save-timeout’, which defaults to the value of
‘auto-save-timeout’.

   To start using this, you need to require the module and turn on
‘ekg-auto-save-mode’ in the ‘ekg-edit-mode’ and ‘ekg-capture-mode’.  For
example:
     (require 'ekg-auto-save)
     (add-hook 'ekg-capture-mode-hook 'ekg-auto-save-mode)
     (add-hook 'ekg-edit-mode-hook 'ekg-auto-save-mode)

   In the capture mode, this.  is equivalent to saving periodically (to
drafts).  In edit mode, it will save the latest version while editing.


File: ekg.info,  Node: Denote,  Next: Apple Notes,  Prev: Auto save,  Up: Extras

18.5 Denote
===========

ekg can export notes as denotes.  Denote is a note taking and file
naming tool.  Primary reason for export is taking backup of your notes
in a git backed repository.  Import is in road map (PR is welcome).

   To export to denote, ‘ekg-denote’ module is required, which is a
separate package from ekg in MELPA.

     (use-package ekg-denote
       :init
       (setq ekg-denote-export-add-front-matter nil)
       (setq ekg-denote-export-backup-on-conflict t)
       (setq ekg-denote-export-title-max-len 50)
       (setq ekg-denote-export-combined-keywords-len 150))

   To export, call ‘ekg-denote-export’ which will export any modified
note after the last export as a denote in the ‘denote-directory’ defined
by ‘denote’ package.  If it is your first time exporting, all the notes
will be exported to the ‘denote-directory’.

   ‘ekg-denote’ keeps record of the last export time in the ekg db and
use that to find the notes changed since last export.  This way the
exports are much faster.  It is suggested to export your notes every
day.

   User can optionally enable adding of denote front-matter to exported
denotes by setting ‘ekg-denote-export-add-front-matter’ to ‘t’.  Denote
front matter is added using ‘denote-add-front-matter’ function defined
by ‘denote’ package which open note in an emacs buffer and requires
manual execution of ‘save-buffer’ by the user.

   Ekg and denote have differences, due to which following customization
are made available:

   • ‘ekg-denote-export-title-max-len’ to trim the title during export.
     Default is 50 characters.
   • ‘ekg-denote-export-combined-keywords-len’ to trim the combined
     length of keywords.  Default is 150 characters.
   • ‘ekg-denote-export-backup-on-conflict’ to backup the denote if both
     ekg and denote are found to be updated after last export.  Default
     is ‘t’.

   It is user responsibility to backup the denotes before and after
export to protect against accidental deletes.  This can be easily done
by keeping denotes in a git repository and making sure to check-in any
changes before and after export.

   If ekg and denote are both found to be updated after the last export
which should ideally not happen, denote is updated with ekg.  A backup
is taken based on ‘ekg-denote-export-backup-on-conflict’ setting.


File: ekg.info,  Node: Apple Notes,  Next: Email,  Prev: Denote,  Up: Extras

18.6 Apple Notes
================

ekg can sync bidirectionally with Apple Notes on macOS.  This is useful
for accessing and editing your notes on an iPhone or iPad, or for
collaborating with others who use Apple Notes.  The sync uses
AppleScript to communicate with Apple Notes in the background, without
bringing the Notes app to the foreground.

   To use this module, you need ‘pandoc’ installed for converting
between org-mode/markdown and HTML (Apple Notes stores content as HTML).

     (require 'ekg-apple-notes)

* Menu:

* Setup::
* Syncing::
* Tags::
* Content conversion::
* Limitations::


File: ekg.info,  Node: Setup,  Next: Syncing,  Up: Apple Notes

18.6.1 Setup
------------

Synced notes are stored in a dedicated Apple Notes folder.  By default
this folder is called "ekg", but you can change it with
‘ekg-apple-notes-folder’.  The folder is created automatically on the
first sync.

   Each ekg database can sync to a different Apple Notes folder.  To
store the folder name in the database itself (so it persists and is
specific to that database), use:

     (ekg-apple-notes-set-folder "my-notes")

   If a folder name is stored in the database, it takes precedence over
the ‘ekg-apple-notes-folder’ customization variable.

   If you have multiple Apple Notes accounts (e.g.  iCloud and a work
account), you can specify which account to use:

     (setq ekg-apple-notes-account "iCloud")

   To limit which notes are exported, set ‘ekg-apple-notes-export-tags’
to a list of tags.  Only notes with at least one of those tags will be
exported:

     (setq ekg-apple-notes-export-tags '("shared" "mobile"))

   Notes with certain tags are excluded from sync entirely.  If
‘ekg-agent’ is loaded, its tag variables (‘ekg-agent-author-tag’ and
‘ekg-agent-self-info-tag’) are automatically excluded, so agent notes
will not be synced.  You can exclude additional tags by customizing
‘ekg-apple-notes-exclude-tags’.


File: ekg.info,  Node: Syncing,  Next: Tags,  Prev: Setup,  Up: Apple Notes

18.6.2 Syncing
--------------

The primary command is ‘ekg-apple-notes-sync’, which performs a
bidirectional sync: first importing from Apple Notes, then exporting to
Apple Notes.  You can also run the two directions separately with
‘ekg-apple-notes-import’ and ‘ekg-apple-notes-export’.

   The sync tracks timestamps so that only notes modified since the last
sync are processed.  A content hash is stored for each exported note so
that imports can distinguish between notes modified in Apple Notes and
notes that are unchanged from the last export.


File: ekg.info,  Node: Tags,  Next: Content conversion,  Prev: Syncing,  Up: Apple Notes

18.6.3 Tags
-----------

ekg tags are stored as individual ‘Tag: <tag>’ lines at the bottom of
each Apple Note.  Each tag gets its own line, preserving the full tag
text including spaces, dots, colons, and other characters.  For example,
a note with the tags "my project" and "date/2024-01-01" would have:

     Tag: my project
     Tag: date/2024-01-01

   These tags don't function as native Apple Notes hashtags, but they
are visible in the note body and can be found via Apple Notes search on
any device.

   Hidden tags (such as "trash" and "draft") are not synced.


File: ekg.info,  Node: Content conversion,  Next: Limitations,  Prev: Tags,  Up: Apple Notes

18.6.4 Content conversion
-------------------------

Note content is converted between org-mode or markdown and HTML using
‘pandoc’.  The path to pandoc can be customized with
‘ekg-apple-notes-pandoc-executable’.

   Links require special handling because Apple Notes strips ‘<a>’ tags
when content is set via AppleScript.  On export, links are converted to
markdown format ‘[text](url)’, which appears as readable plain text in
Apple Notes (and plain URLs are auto-linked by Apple Notes in its UI).
On import back to org-mode, these are reconstructed as ‘[[url][text]]’
org links.


File: ekg.info,  Node: Limitations,  Prev: Content conversion,  Up: Apple Notes

18.6.5 Limitations
------------------

   • *Links*: Named hyperlinks degrade to ‘[text](url)’ plain text in
     Apple Notes.  Plain URLs are auto-linked by Apple Notes.  On
     re-import, a heuristic reconstructs the links.
   • *Formatting*: Apple Notes normalizes HTML aggressively (e.g.,
     ‘<h1>’ becomes bold text with a larger font size, ‘<ol>’ becomes
     ‘<ul>’).  Repeated round-trips may cause minor formatting drift.
   • *Attachments*: Images and other attachments in Apple Notes are not
     synced.
   • *Deletion*: Deleting a note in Apple Notes does not automatically
     delete it in ekg (and vice versa).  Deletion sync may be added in
     the future.


File: ekg.info,  Node: Email,  Next: Org Tasks,  Prev: Apple Notes,  Up: Extras

18.7 Email
==========

The ‘ekg-email’ module adds schema for storing the ‘to’, ‘from’, and
‘cc’ email addresses associated with an email, and a way to import an
email into notes from Gnus.  Other importing methods may be available in
the future.

   While in a Gnus article buffer, you can execute the command
‘ekg-email-from-gnus’.

   Once mails are imported, you can search for email addresses
associated with them with ‘ekg-email-show-notes-with-address’.


File: ekg.info,  Node: Org Tasks,  Next: Agent,  Prev: Email,  Up: Extras

18.8 Org Tasks
==============

The ‘ekg-org’ module integrates ekg with ‘org-mode’, allowing ekg notes
to serve as a source of tasks for the org agenda and providing a
dedicated task view.  This is not a substitute for ‘org-mode’ -- it is a
lightweight, experimental alternative designed for situations where a
human and an AI agent need to collaborate on a set of tasks, with either
party able to create, update, or reorganize tasks at any time.  The
module includes full org-agenda integration, so tasks managed in ekg
appear alongside your regular org tasks.

   ‘ekg-org’ is part of the main ‘ekg’ package and does not need to be
installed separately, but it does need to be required:

     (require 'ekg-org)

* Menu:

* Data model::
* Org agenda integration::
* Capturing tasks::
* Changing task state::
* The ekg-org-view::
* Agent integration::
* Command-line tools::


File: ekg.info,  Node: Data model,  Next: Org agenda integration,  Up: Org Tasks

18.8.1 Data model
-----------------

Tasks in ‘ekg-org’ are ordinary ekg notes with special tags and
properties:

   • *Tags*: Every task has the ‘org/task’ tag.  Its TODO state is
     stored as a tag with the prefix ‘org/state/’ (e.g.,
     ‘org/state/todo’, ‘org/state/done’).  Archived tasks additionally
     carry the ‘org/archive’ tag.  Any other tags are preserved as
     regular ekg tags.

   • *Title*: Each task must have a title (stored in the ‘:titled/title’
     property), which becomes the org headline when rendered.

   • *Hierarchy*: Tasks can have parent-child relationships via the
     ‘:org/parent’ property (an integer note ID). Children are
     discovered through a virtual reversed property ‘:org/children’.
     There is no limit to nesting depth.

   • *Scheduling*: Tasks can have ‘:org/deadline’ and ‘:org/scheduled’
     properties (stored as unix timestamps), which appear as standard
     org timestamps.

   • *Ordering*: Sibling tasks are ordered by the ‘:org/sort-order’
     property (defaulting to creation time when unset).

   • *Org properties*: Arbitrary org-style properties (uppercase
     key/value pairs following org property syntax) can be attached to
     tasks via the ‘:org/property’ property, and manipulated with
     ‘ekg-org-get-property’, ‘ekg-org-set-property’, and
     ‘ekg-org-remove-property’.


File: ekg.info,  Node: Org agenda integration,  Next: Capturing tasks,  Prev: Data model,  Up: Org Tasks

18.8.2 Org agenda integration
-----------------------------

When ‘ekg-org’ is loaded, it registers a virtual file ‘/ekg:tasks.org’
with the org agenda.  This file is generated on the fly from your ekg
tasks and is automatically added to ‘org-agenda-files’.  No actual file
is written to disk; a file-name handler intercepts Emacs's file
operations and returns the generated content.

   This means that ‘M-x org-agenda’ will include your ekg tasks
alongside any other org files you use.  The virtual file is re-generated
whenever it is read, and its modification time tracks the ekg database
file, so the agenda picks up changes automatically.

   Archived tasks are served from a separate virtual file
‘/ekg:tasks.org_archive’.


File: ekg.info,  Node: Capturing tasks,  Next: Changing task state,  Prev: Org agenda integration,  Up: Org Tasks

18.8.3 Capturing tasks
----------------------

You can capture a new org task interactively with ‘ekg-org-capture’,
which prompts for a title and creates a new note pre-tagged with
‘org/task’ and ‘org/state/todo’.  You can also create tasks
programmatically:

     (ekg-save-note
      (ekg-note-create
       :text "Description of the work."
       :mode 'org-mode
       :tags '("org/task" "org/state/todo" "my-project")
       :properties '(:titled/title ("Implement feature X"))))


File: ekg.info,  Node: Changing task state,  Next: The ekg-org-view,  Prev: Capturing tasks,  Up: Org Tasks

18.8.4 Changing task state
--------------------------

The command ‘ekg-org-change-state’ (available in note editing buffers
and ‘ekg-notes-mode’ buffers) prompts for a new TODO state from
‘org-todo-keywords-1’ and updates the task's state tag accordingly.


File: ekg.info,  Node: The ekg-org-view,  Next: Agent integration,  Prev: Changing task state,  Up: Org Tasks

18.8.5 The ekg-org-view
-----------------------

‘ekg-org-view-mode’ is a read-only, interactive view of your task
hierarchy, built with the ‘vui’ library.  It presents tasks as an
indented outline that looks like an org buffer (with fontified headings,
TODO keywords, and tags) but is backed entirely by the ekg database.

   To open the view, call ‘ekg-org-view’.  For archived tasks, call
‘ekg-org-archive-view’.  To focus on a single task and its subtree, call
‘ekg-org-view-task’.

   The view refreshes automatically whenever any ekg note is saved, so
changes made by you, by an agent, or by the command-line tools are
reflected immediately.

   The keybindings are modeled after org-mode's speed keys (the
single-letter shortcuts available when point is at the beginning of a
headline), so they should feel familiar to org-mode users.

  1. Navigation

     The following keybindings are available in ‘ekg-org-view-mode’:

     Key   Command                           Description
     ----------------------------------------------------------------------------------
     ‘n’   ‘ekg-org-view-next-heading’       Move to the next heading
     ‘p’   ‘ekg-org-view-previous-heading’   Move to the previous heading
     ‘f’   ‘ekg-org-view-next-sibling’       Move to the next same-level heading
     ‘b’   ‘ekg-org-view-previous-sibling’   Move to the previous same-level heading
     ‘u’   ‘ekg-org-view-up-heading’         Move to the parent heading

  2. Actions

     Key         Command                          Description
     -------------------------------------------------------------------------------------------------
     ‘t’         ‘ekg-org-view-cycle-state’       Change the TODO state
     ‘TAB’       ‘ekg-org-view-toggle-collapse’   Collapse or expand the subtree
     ‘RET’       ‘ekg-org-view-open-note’         Open the note for editing in ekg
     ‘c’         ‘ekg-org-view-create-child’      Create a child task under point
     ‘C’         ‘ekg-org-view-create-sibling’    Create a sibling task after point
     ‘a’         ‘ekg-org-view-archive’           Archive the task at point
     ‘d’         ‘ekg-org-view-delete’            Trash (soft-delete) the task
     ‘L’ / ‘l’   ‘ekg-org-view-promote’           Promote: make task a sibling of its parent
     ‘R’ / ‘r’   ‘ekg-org-view-demote’            Demote: make task a child of its previous sibling
     ‘U’         ‘ekg-org-view-move-up’           Move task up (swap with previous sibling)
     ‘D’         ‘ekg-org-view-move-down’         Move task down (swap with next sibling)
     ‘g’         ‘ekg-org-view-refresh’           Manually refresh the view

  3. How it differs from org-mode

     ‘ekg-org-view’ is intentionally simpler than a full org-mode
     buffer:

        • Tasks are stored in the ekg database, not in files.  This
          makes them accessible to agents, command-line tools, and
          programmatic APIs without needing to parse org syntax.
        • The view is read-only.  To edit a task's body text, press
          ‘RET’ to open it in ekg's normal note-editing buffer.
        • There is no in-buffer text editing, column view, clocking, or
          other advanced org features.  The focus is on task
          organization (creating, reordering, re-parenting, and state
          changes).
        • Because tasks live in the database, both a human in Emacs and
          an external agent can modify tasks concurrently.  The view
          auto-refreshes to reflect all changes.


File: ekg.info,  Node: Agent integration,  Next: Command-line tools,  Prev: The ekg-org-view,  Up: Org Tasks

18.8.6 Agent integration
------------------------

When ‘ekg-agent’ is loaded alongside ‘ekg-org’, the agent gains tools
for working with org tasks:

   • *add_org_task*: Create a new task with a title, description, tags,
     optional parent, state, deadline, and scheduled date.
   • *set_org_task_status*: Change the TODO state of an existing task.
   • *list_org_items*: List all active tasks, optionally filtered by
     state.

   These tools are registered automatically by ‘ekg-org-initialize’ when
‘ekg-agent’ is available.  This allows the agent to create sub-tasks,
mark items done, and review your task list as part of its autonomous
operation.


File: ekg.info,  Node: Command-line tools,  Prev: Agent integration,  Up: Org Tasks

18.8.7 Command-line tools
-------------------------

Two command-line scripts are provided in ‘agent_tools/’ for use by
external agents:

   • ‘ekg-org-add-child’: Create a child task under an existing parent.
     Supports ‘--title’, ‘--note’, ‘--tag’, ‘--status’, and ‘--property
     KEY=VALUE’ options.
   • ‘ekg-org-edit’: Edit an existing task's title, content, status, or
     org properties.  Supports ‘--title’, ‘--note’, ‘--append’,
     ‘--status’, ‘--set-property KEY=VALUE’, and ‘--remove-property KEY’
     options.

   These complement the general ‘ekg-add-note’ and ‘ekg-read’ scripts.
See the ‘agent_tools/skill.md’ file for complete usage documentation.


File: ekg.info,  Node: Agent,  Prev: Org Tasks,  Up: Extras

18.9 Agent
==========

The ‘ekg-agent’ module provides an autonomous agent that can help you
with your notes.  This agent can review your notes and org-mode agenda,
searching across notes by embeddings or tags as appropriate, and
proactively create new notes to help you, such as offering advice,
pointing out connections, or suggesting resources.

   ‘ekg-agent’ is a separate package from ‘ekg’ and must be installed
separately.

     (use-package ekg-agent
       :init
       (setq ekg-agent-code-command "~/.local/bin/claude -p --dangerously-skip-permissions")
       (add-to-list 'ekg-agent-extra-tools (list ekg-agent-tool-run-elisp
                                                 ekg-agent-tool-code)))

   To use this module, you must have an LLM provider configured as
described in the *note LLM: LLM. section.  You can then load the agent:

     (require 'ekg-agent)

   The primary command is ‘ekg-agent-evaluate-status’, which runs a
single session where the agent reviews your recent activity and decides
if it can be of assistance.  It can create notes to you, or to itself,
to help it work.

   You can also ask the agent to help you with a specific note you are
writing or editing.  The command ‘ekg-agent-note-response’ (bound to
‘C-c .’ by default, overriding the ‘ekg-llm’ binding) will start an
agent loop focused on your current note.  This is an agentic version of
‘ekg-llm-send-and-append-note’.  The agent can use tools to research
your other notes and decide whether to append information to your note
or replace it entirely.  As with other LLM commands, a prefix argument
allows you to edit the instructions sent to the agent.

   You can also ask the agent a direct question at any time with the
command ‘ekg-agent-ask’.  This will prompt you for a question, which is
then sent to the agent.  To provide context, ekg will automatically
include lightweight information about the current buffer and short
previews of your last 10 notes.  The current buffer contents are not
sent up front; the agent can use tools such as ‘search_buffer’ or
‘read_buffer’ when it needs specific text.  The agent will be able to
search your notes and use any other tools available.  It will then
provide its response either by displaying it in a popup buffer or by
creating a new note, choosing the method it deems most appropriate.
Once the agent has provided a response, its task for that question is
complete.  For example:

     (ekg-agent-ask "What are the main themes in my recent notes on 'systems thinking'?")

   This allows for quick, ad-hoc interactions with the agent without
needing to be in a specific note buffer.  There's a similar
‘ekg-agent-ask-with-region’ when you explicitly want to send a selected
region as context.

   You can also ask a question about a specific note with
‘ekg-agent-ask-with-note’.  When called from an ‘ekg-notes-mode’ buffer,
it uses the note at point; when called from a note editing buffer, it
uses that note.  You can also pass a note ID programmatically.

   When ‘ekg-llm-provider’ is a list, the ‘ekg-agent-ask’,
‘ekg-agent-ask-with-note’, and ‘ekg-agent-ask-with-region’ commands can
be called with a prefix argument (normally via ‘C-u M-x’) to select the
provider for that agent session.  Without a prefix argument, the agent
chooses the first configured provider with tool calling support, falling
back to the first provider in the list.  When ‘ekg-llm-provider’ is a
single provider, the prefix argument does not prompt for a provider.

   Internal agent notes (using the ‘agent/self-info’ and
‘agent/instruct’ tags) are hidden from normal view by being added to
‘ekg-hidden-tags’.  You can view these notes specifically with
‘ekg-agent-show-internal-notes’.

   You can also have the agent run automatically every day.  To do this,
customize ‘ekg-agent-daily-time’ to your preferred time (default is
"09:00"), and then run:

     (ekg-agent-schedule-daily)

   This will set up a timer to run the agent daily.  You can cancel this
schedule with ‘ekg-agent-cancel-daily’.

   The agent uses specific tags to organize its work:
   • ‘ekg-agent-author-tag’ (default "agent"): All notes created by the
     agent will have this tag.
   • ‘ekg-agent-self-info-tag’ (default "agent/self-info"): The agent
     uses this tag to store information for itself about your
     preferences or its own past actions.

   You can customize these tags if you prefer different names.

   You can also customize the tools available to the agent by modifying
‘ekg-agent-extra-tools’.  This allows you to add your own tools to the
standard set of tools provided by the agent.  We provide two tools that
are dangerous but greatly improve the power of the agent:
‘ekg-agent-tool-run-elisp’, which enables the agent to run arbitrary
elisp commands, and ‘ekg-agent-tool-code’, which enables the agent to
integrate with coding tools (Claude Code, Codex, Goose, etc).  To use
these tools effectively it's important to run without permissions.
Here's an example:

     (setq ekg-agent-code-command "~/.local/bin/claude -p --dangerously-skip-permissions")
     (setq ekg-agent-extra-tools (list ekg-agent-tool-run-elisp ekg-agent-tool-code))

   By default, the coding tool prompt is sent on standard input so
prompt contents do not appear in process listings.  If your command
requires the prompt as its final command-line argument, set
‘ekg-agent-code-command-prompt-method’ to ‘argument’.

* Menu:

* Optional connected-data tool libraries::
* Agent log mode continuing and cancelling::
* AGENTS.md support: AGENTSmd support.
* Command-line tools for agent integration::


File: ekg.info,  Node: Optional connected-data tool libraries,  Next: Agent log mode continuing and cancelling,  Up: Agent

18.9.1 Optional connected-data tool libraries
---------------------------------------------

Some agent tools are packaged as optional libraries because they expose
data from local connected services.  These tools are not enabled by
default.  Require the library you want and add its tool list to
‘ekg-agent-extra-tools’.

   The Gnus tool library provides tools for listing folders, fetching
latest counts, searching messages, reading message headers, and reading
full messages by an opaque ‘message_id’ returned from header and search
tools.  Search prompts are disabled while the tools run so a backend
prompt fails instead of blocking the agent.

     (require 'ekg-agent-tools-gnus)
     (setq ekg-agent-extra-tools
           (append ekg-agent-tools-gnus-tools ekg-agent-extra-tools))

   For example, an agent can search for recent messages from a sender
whose content matches a term by calling ‘gnus_search_messages’ with
structured arguments such as ‘from’ set to "Stefan Monnier" and ‘body’
set to "uuid".  It can then pass the returned ‘message_id’ to
‘gnus_read_message’ to read the message.

   The Elfeed tool library provides tools for listing feeds, searching
entries using Elfeed filter syntax, and reading one entry by ID or link.

     (require 'ekg-agent-tools-elfeed)
     (setq ekg-agent-extra-tools
           (append ekg-agent-tools-elfeed-tools ekg-agent-extra-tools))

   The standard tool set also includes read-only Emacs reference tools.
The agent can use ‘emacs_help_search’ and ‘emacs_help_symbol’ to search
and display Emacs help for functions, macros, variables, faces, and
other symbols.  It can use ‘emacs_info_search’ and ‘emacs_info_node’ to
search and read the installed Info manuals, including the Emacs, Emacs
Lisp, and Common Lisp extension manuals.  These tools let the agent look
up Emacs and Elisp behavior directly instead of guessing from memory.


File: ekg.info,  Node: Agent log mode continuing and cancelling,  Next: AGENTSmd support,  Prev: Optional connected-data tool libraries,  Up: Agent

18.9.2 Agent log mode: continuing and cancelling
------------------------------------------------

When the agent starts a session, it creates a log buffer (named
according to ‘ekg-agent-log-buffer-name-format’) and enables
‘ekg-agent-log-mode’, a minor mode that provides commands for
controlling the agent.  The mode-line lighter shows ‘Agent:run’ while
the agent is active and ‘Agent:idle’ when it has stopped.

   The following commands are available in the log buffer:

   • ‘C-c C-c’ (‘ekg-agent-continue’): Continue the agent from where it
     left off.  The agent's full conversation history is preserved, so
     continuing resumes exactly where the previous session
     stopped--whether it completed normally, timed out, or was
     cancelled.  With a prefix argument (‘C-u C-c C-c’), you are
     prompted for a message to send to the agent, which lets you give it
     new instructions or redirect its work.

   • ‘C-c C-k’ (‘ekg-agent-cancel’): Cancel the current agent run.  The
     agent will stop after the current in-flight LLM call completes (it
     does not interrupt a call in progress).  The conversation prompt is
     preserved in the buffer, so you can resume later with ‘C-c C-c’.

   • ‘C-c C-d’ (‘ekg-agent-export-debug-json’): Export the stored
     prompt, conversation interactions, tool definitions, successful
     tool history, completion requirements, configuration, and visible
     log text to a temporary JSON file.  The file path is copied to the
     kill ring and can be analyzed with the
     ‘agent_tools/ekg-agent-debug-analyzer’ skill.

   This makes it easy to supervise long-running agent sessions: you can
cancel if the agent is going in the wrong direction, give it new
guidance, and continue.

   For long-running work, the agent is also reminded to call
‘summarize_state’ if it has gone too long without an explicit progress
update.  You can control this with ‘ekg-agent-status-reminder-seconds’.


File: ekg.info,  Node: AGENTSmd support,  Next: Command-line tools for agent integration,  Prev: Agent log mode continuing and cancelling,  Up: Agent

18.9.3 AGENTS.md support
------------------------

The agent automatically reads ‘AGENTS.md’ files to pick up user-specific
instructions and preferences.  Two locations are checked when the
agent's context is built:

  1. ‘~/AGENTS.md’ -- your home directory, for global preferences that
     apply to all projects.
  2. ‘<default-directory>/AGENTS.md’ -- the current directory when the
     agent is started, for project-specific instructions.

   If either file exists, its contents are appended to the agent's
system prompt.  If both exist (and are in different directories), both
are included.

   In addition, the agent has access to the ‘read_agents_md’ tool, which
lets it read the ‘AGENTS.md’ file from any directory.  This is part of
the standard tool set and requires no configuration.  It allows the
agent to discover instructions for a project directory it encounters
during its work, even if that directory was not the starting directory.


File: ekg.info,  Node: Command-line tools for agent integration,  Prev: AGENTSmd support,  Up: Agent

18.9.4 Command-line tools for agent integration
-----------------------------------------------

The ‘ekg’ package provides command-line tools in the ‘agent_tools/’
subdirectory that enable external AI agents to use ekg as persistent
memory storage.  These tools communicate with the running Emacs daemon
via ‘emacsclient’ and are designed for programmatic use by agents.

   If you use ‘ekg’ with an agent, you need to ‘(require 'ekg-agent)’.

  1. Agent tool location

     The command-line tools are located in the ‘agent_tools/’
     subdirectory of the ekg installation:

        • ‘agent_tools/ekg-add-note’ - Create notes
        • ‘agent_tools/ekg-read’ - Read notes by tag, ID, or search
        • ‘agent_tools/ekg-tags’ - Search and list tags
        • ‘agent_tools/ekg-update-tags’ - Add or remove tags on existing
          notes
        • ‘agent_tools/ekg-trash’ - Soft-delete notes

     To use these tools, agents should locate the ekg installation
     directory and reference the scripts with their full path, for
     example: ‘$EKG_DIR/agent_tools/ekg-add-note’

  2. Skill file

     The file ‘agent_tools/skill.md’ is a complete reference document
     describing all available tools, tag conventions, and recommended
     workflows.  It is designed to be used as an agent skill -- a file
     that an agent reads to learn how to use ekg.

     To make this skill available to your agent, symlink ‘skill.md’ into
     wherever your agent stores its skill files.  For example:

          ln -s /path/to/ekg/agent_tools/skill.md ~/.my-agent/skills/ekg/SKILL.md

     This way, the agent always has up-to-date documentation for the ekg
     tools without needing to duplicate it.

     The directory ‘agent_tools/ekg-agent-debug-analyzer/’ contains a
     separate skill for investigating JSON exports from
     ‘ekg-agent-export-debug-json’.  It includes a script that produces
     a first-pass Markdown report of inefficient tool use, errors,
     repeated calls, oversized tool results, completion blockers, and
     missing required actions.  Symlink its ‘SKILL.md’ into an agent
     skill directory when you want an external agent to improve
     ‘ekg-agent’ behavior from exported sessions.

  3. Instructions for configuring agents

     When setting up an AI agent to use ekg for memory, the recommended
     approach is to provide the agent with the ‘skill.md’ file as
     described above.  For agents that do not support skill files, you
     can provide these instructions directly:

       1. Long version

               You have access to an emacs-based external memory system called ekg.  It may contain
               both notes the user has written, but is also accessible for you to read and write to.
               To store information for later retrieval, use the ekg-add-note tool located at:
               <ekg-installation-path>/agent_tools/ekg-add-note.  It requires a running emacs
               with the daemon running (via the emacs `server-start` command), if there is an issue
               connecting, alert the user and ask them to run `server-start`.

               Usage:
                 ekg-add-note --tag <tag1> --tag <tag2> --note "<note text>" --mode org-mode

               Parameters:
                 --tag TAG    : Categorize the note (can be used multiple times)
                 --note TEXT  : The content to store (required)
                 --mode MODE  : Format (org-mode, markdown-mode, or text-mode; default: org-mode)

               The system will automatically:
               - Mark the note as agent-created (adds "agent" tag)
               - Apply automatic tagging functions (e.g., adds current date as "date/YYYY-MM-DD")
               - Format the content with appropriate markers
               - Store timestamps for creation and modification

               Tags may contain spaces or other special characters.  Tags can be hierarchical, some are standard, such as "project/revise event processing/design"

               Example usage:
                 ekg-add-note --tag "project/revise event processing" --tag "progress" \\
                   --note "Completed database optimization. 40% performance improvement."

               Use tags strategically:
               - Use tag `project/<name>` for project-specific information.
               - Use descriptive tags for easy retrieval.
               - Multiple tags and nested tags help organize related information.
               - Use `agent/instruct` to tag and retrieve information to change the agent behavior.

               To retrieve information from your ekg memory, use the ekg-read tool located at:
               <ekg-installation-path>/agent_tools/ekg-read

               The tool returns JSON-formatted results for easy parsing.

               Usage modes:

               1. Search by tags (AND logic - notes must have all specified tags):
                  ekg-read --tag <tag1> --tag <tag2> -n 10 -w 100

               2. Retrieve specific note by ID:
                  ekg-read <note-id>

               3. Semantic search (finds conceptually similar notes):
                  ekg-read --semantic-search "machine learning optimization"

               4. Text search (keyword matching):
                  ekg-read --search "deadline"

               Parameters:
                 --tag TAG            : Filter by tag (can be used multiple times for AND logic)
                 -n, --num NUM        : Max notes to return (default: 10)
                 -w, --words NUM      : Max words per note (default: 100)
                 --semantic-search Q  : Find conceptually related notes
                 --search QUERY       : Find notes containing keywords

               Output format (JSON):
               [
                 {
                   "id": 12345678,
                   "text": "Note content...",
                   "mode": "org-mode",
                   "tags": ["tag1", "tag2"],
                   "creation_time": 1768889121,
                   "modified_time": 1768889121
                 }
               ]

               Example workflow:
                 # Retrieve recent project notes
                 ekg-read --tag "project/current-work" -n 5

                 # Find notes about specific topic
                 ekg-read --search "optimization" -n 10

                 # Get exact note by ID
                 ekg-read 12345678

               You should use ekg memory to:
               1. Record important findings, decisions, and progress
               2. Build up context across multiple interactions
               3. Remember user preferences and past discussions
               4. Track long-running projects and goals

               Workflow:
               - At the end of significant interactions, store key information
               - Before starting new tasks, retrieve relevant context
               - Use tags to organize information hierarchically
               - Prefer specific, actionable notes over vague summaries

               Example:
                 # Store a finding
                 ekg-add-note --tag "user-preferences" --tag "coding-style" \\
                   --note "User prefers descriptive variable names over short abbreviations"

                 # Later, retrieve preferences
                 ekg-read --tag "user-preferences" -n 10

               Additional tools:

                 # Search and list tags
                 ekg-tags                              # All tags
                 ekg-tags --co-tagged prompt           # Tags co-occurring with "prompt"
                 ekg-tags --regex "org/state/.*"       # Tags matching regex
                 ekg-tags --prefix "project/"          # Tags with prefix

                 # Update tags on a note
                 ekg-update-tags <note-id> --add "done" --remove "in-progress"

                 # Soft-delete a note
                 ekg-trash <note-id>


       2. Shorter version

               You have access to an emacs-based external memory system called ekg, which stores notes from users and agents with tags.  Use it as a memory when working.  It requires a running emacs
               with the daemon running (via the emacs `server-start` command), if there is an issue
               connecting, alert the user and ask them to run `server-start`.

               To store information for later retrieval, use the ekg-add-note tool located at:
               <ekg-installation-path>/agent_tools/ekg-add-note

               Usage:
                 ekg-add-note --tag <tag1> --tag <tag2> --note "<note text>" --mode org-mode

               Tags may contain spaces or other special characters.  Use tags with the "project/" prefix to store information about projects.  Use the `agent/instruct` tag to change future behavior of the agent.

               To retrieve information from your ekg memory, use the ekg-read tool located at:
               <ekg-installation-path>/agent_tools/ekg-read

               The tool returns JSON-formatted results.

               Usage modes:

               1. Search by tags (AND logic - notes must have all specified tags):
                  ekg-read --tag <tag1> --tag <tag2> -n 10 -w 100

               2. Retrieve specific note by ID:
                  ekg-read <note-id>

               3. Semantic search (finds conceptually similar notes):
                  ekg-read --semantic-search "machine learning optimization"

               4. Text search (keyword matching):
                  ekg-read --search "deadline"

               To search and list tags, use ekg-tags:
                 ekg-tags                              # All tags
                 ekg-tags --co-tagged prompt           # Tags co-occurring with "prompt"
                 ekg-tags --regex "pattern"            # Tags matching regex

               To update tags on a note: ekg-update-tags <note-id> --add tag1 --remove tag2
               To soft-delete a note: ekg-trash <note-id>

               You should use ekg memory to:
               1. Record important findings, decisions, and progress
               2. Build up context across multiple interactions
               3. Remember user preferences and past discussions
               4. Track long-running projects and goals

               Workflow:
               - At the end of significant interactions, store key information
               - Before starting new tasks, retrieve relevant context
               - Use tags to organize information hierarchically
               - Prefer specific, actionable notes over vague summaries

  4. Technical details

     For system administrators configuring agent access:

       1. Requirements

             • An Emacs daemon must be running for the tools to function
             • Start daemon: ‘emacs --daemon’
             • For named daemons: ‘emacs --daemon=<name>’ and use
               ‘--daemon <name>’ option

       2. Implementation

             • Tools use ‘ekg-agent-add-note’ and ‘ekg-agent-read-notes’
               elisp functions
             • Communication via ‘emacsclient’ for fast execution (~0.1
               seconds per operation)
             • Notes automatically tagged with ‘ekg-agent-author-tag’
               (default: "agent")
             • Text automatically wrapped in LLM output markers for
               proper attribution
             • JSON encoding via Emacs ‘json-encode’ function

       3. Elisp API

          For direct elisp integration:

               ;; Add a note
               (ekg-agent-add-note "Note content" '("tag1" "tag2") 'org-mode)

               ;; Read notes by tags
               (ekg-agent-read-notes :tags '("tag1" "tag2") :num 10 :max-words 100)

               ;; Read by note ID
               (ekg-agent-read-notes :note-id 12345678)

               ;; Semantic search
               (ekg-agent-read-notes :semantic-search "query" :num 10)

               ;; Text search
               (ekg-agent-read-notes :text-search "query" :num 10)

               ;; List all tags
               (ekg-tags)

               ;; Tags with prefix
               (ekg-tags-with-prefix "project/")

               ;; Tags containing substring
               (ekg-tags-including "moltbot")

               ;; Tags co-occurring with prompt tag
               (ekg-agent-cotagged-prompt-tags)


File: ekg.info,  Node: Design,  Prev: Extras,  Up: Top

19 Design
*********

* Menu:

* The triple database::
* The metadata section::


File: ekg.info,  Node: The triple database,  Next: The metadata section,  Up: Design

19.1 The triple database
========================

The ekg package uses the triples package to interface with a sqlite
database.  The reason a database is useful, even for text, is because
databases are extremely fast, very flexible, and extremely easy to
change.  In general, the less your data looks like just files with text
in them, the more a database makes sense.  In ekg, we can separate the
notion of tags from the text, which makes writing functions such as
‘ekg-global-rename-tag’ trivial, and the execution extremely fast.

   The decision to use the triples package, though, is related to a
different design choice.  In a triple-based system, there's only one
database table with four columns, a ‘subject’, ‘predicate’, ‘object’,
and ‘properties’.  One way to think of this schema is that it defines
links of different types from a subject to an object.  This is combined
with a schema, itself defined in triples.  The triples define that
subjects can have types, and those types can have properties.  Those
properties are expressed in this triple format.  In ekg, the subjects
correspond to the IDs of the notes, or tags.  Subjects can have multiple
types, and data is factored into types that belong together, with a
specific meaning.  To give an example, listing out the data for a note
might look something like:

     33204698034|base/type|tagged|()
     33204698034|tagged/tag|"date/2022-11-06"|(:index 0)
     33204698034|tagged/tag|"lentil stew"|(:index 1)
     33204698034|base/type|text|()
     33204698034|text/text|"Made a great lentil stew with dried porcini mushrooms and delicata squash."
     33204698034|text/mode|org-mode|()
     33204698034|base/type|time-tracked|()
     33204698034|time-tracked/creation-time|1667787928|()
     33204698034|time-tracked/modified-time|1667787986|()

   In this example, ‘33204698034’ is the ID for this note.  It has a
type (‘base/type’), of ‘tagged’, which means this is something that has
tags.  The tags are a list, so the properties contain their index in the
list.  Because each one is stored individually, we can easily find all
entities with each tag, by querying on all subjects with a particular
object value.  This is how reverse links work in the triples package.
In this case, there are two tags, "date/2022-11-06", and "lentil stew".
The note comes from another type, ‘text’.  And yet another important
property, the modification time, is on yet another type, ‘time-tracked’.
These are all independent.  It is possible to have subjects that have
tags but not text, although this doesn't happen currently in ekg.  It's
also possible to have any object have a creation and modified time.

   Using a triples scheme has the advantage that it is very easy to
integrate with.  All data is very "flat", without having to worry about
tables and their schemas.  The uniformity means that it lends itself
well to integrations, which typically would provide a new type and new
data.  The disadvantage is that it is typically less efficient to query,
at least for more complicated queries.  On databases that typically will
be used with ekg, this should be not noticeable.

   IDs, stored as subjects, can be resources.  This is useful when we
want to store data about some unique thing, such as an URL.  Because
triples define a graph, every object can be a subject.  For an example,
if some data in the graph has a value of "<http://emacs.org>", then we
can attach more data to that value, such as tags, notes or anything
else.  This is how we store notes about web pages (‘ekg-capture-url’).
Having IDs that are meaningful is also useful to enforce unique data,
and force that data isn't duplicated.  For example, with this design,
you couldn't have both a "tag" entity and a "page" entity that are
separate: if they are the same object value, they will be the same
subject, with the same ID.  This leads, in our opinion, to a better
design.  Also it's useful to note that IDs can be anything, even
different types of objects.  Integers, strings, symbols.  This is
useful, because objects can be anything.  Because of the design of the
triple database, all data can be expanded on with their own data, and
that data itself expanded on.  This seems like a useful property to have
for a personal knowledge system.


File: ekg.info,  Node: The metadata section,  Prev: The triple database,  Up: Design

19.2 The metadata section
=========================

Because the user may want to modify or create both the text and other
database properties at the same time, we use a single buffer that lets
the user do both.  Because of this design choice, we have to divide the
buffer up into two sections: a metadata section and the text section.
The metadata section is on top, and has a specific format.  Because of
this, some ‘org-mode’ functions may not work correctly, because they
assume the whole buffer is an org-mode file.  Without this design,
however, it isn't clear how the user could easily see and modify
everything they need.  Theoretically, having another window might work,
but this would add other complications: the user might not want several
windows, the user might select or bury one of them, and more.  There
isn't an obvious ideal solution.  It's possible that the design of the
capture/edit buffer may change in the future to fix some of the issues
we see with the current implementation.



Tag Table:
Node: Top207
Node: Introduction2979
Node: Installation3709
Node: Sqlite dependency3908
Node: Installing via Melpa4289
Node: Installing by hand5156
Node: Changelog5861
Node: Version 0926658
Node: Version 0916835
Node: Version 0907249
Node: Version 0807995
Node: Version 0718404
Node: Version 0708592
Node: Version 0649109
Node: Version 0639771
Node: Version 06210267
Node: Version 06110519
Node: Version 06010905
Node: Version 05111346
Node: Version 0511718
Node: Version 04313589
Node: Version 04213817
Node: Version 04114363
Node: Version 0414761
Node: Version 03316445
Node: Version 03216697
Node: Version 03116913
Node: Version 0318553
Node: Version 02119992
Node: Version 0220349
Node: Database21811
Node: Concepts and data model in ekg22361
Node: Understanding and editing the note buffer24166
Node: Drafts30046
Node: A warning about org-mode30770
Node: Capturing notes31456
Node: Templates34312
Node: Changing the initial tags of a note35899
Node: Inline commands36584
Node: Viewing tags or notes42735
Node: Commands in the notes buffer44822
Node: Customizing note display in ekg-notes-mode47177
Node: Searching for notes48103
Node: Magic tags49504
Node: The trash51955
Node: Links to ekg in org-mode52706
Node: Importing from org-roam53293
Node: Backups54920
Node: Database maintenance55749
Node: Customizing ekg56598
Node: Hidden Tags57830
Node: Text Truncation Method58390
Node: Integration with ekg60288
Node: Extras62131
Node: Embeddings62529
Node: Logseq67241
Node: Exporting69236
Node: Importing72202
Node: LLM73464
Node: Augmenting notes with LLM output75352
Node: Using ekg notes as prompts78815
Node: Querying your ekg database80488
Node: Auto save81439
Node: Denote82445
Node: Apple Notes84929
Node: Setup85615
Node: Syncing86974
Node: Tags87618
Node: Content conversion88286
Node: Limitations88985
Node: Email89771
Node: Org Tasks90340
Node: Data model91314
Node: Org agenda integration92832
Node: Capturing tasks93691
Node: Changing task state94306
Node: The ekg-org-view94687
Ref: Navigation95668
Ref: Actions96327
Ref: How it differs from org-mode97619
Node: Agent integration98491
Node: Command-line tools99283
Node: Agent100115
Node: Optional connected-data tool libraries105925
Node: Agent log mode continuing and cancelling107989
Node: AGENTSmd support110158
Node: Command-line tools for agent integration111284
Ref: Agent tool location111858
Ref: Skill file112533
Ref: Instructions for configuring agents113693
Ref: Long version114000
Ref: Shorter version119430
Ref: Technical details122093
Ref: Requirements122180
Ref: Implementation122448
Ref: Elisp API122987
Node: Design123900
Node: The triple database124038
Node: The metadata section128453

End Tag Table


Local Variables:
coding: utf-8
End:
