= remwharead(1)
:doctype: manpage
:Author: tastytea
:Email: tastytea@tastytea.de
:Date: 2020-10-31
:Revision: 0.0.0
:man source: remwharead
:man manual: General Commands Manual
== NAME
remwharead - Saves URIs of things you want to remember in a database
== SYNOPSIS
*remwharead* [*-t*=_tags_] [*-N*] _URI_
*remwharead* *-e*=_format_ [*-f*=_file_] [*-T*=_start_,_end_] [[*-s*|*-S*]=_expression_] [*-r*]
*remwharead* [*-d*=_URI_]
== DESCRIPTION
*remwharead* saves URIs of things you want to remember in a database along with
an URI to the archived version, the current date and time, title, description,
the full text of the page and optional tags.
The database can be filtered by time, tags and full text and exported to CSV,
AsciiDoc, a bookmarks file, JSON, RSS, a list of hyperlinks or a rofi-compatible
list.
Archiving is done using the Wayback machine from the
https://archive.org/[Internet Archive].
== OPTIONS
*-t*=_tags_, *--tags*=_tags_::
Add tags to _URI_, delimited by commas.
*-e*=_format_, *--export*=_format_::
Export to _format_. Possible values are _csv_, _asciidoc_, _bookmarks_,
_simple_, _json_, _rss_, _link_ or _rofi_. See _FORMATS_.
*-f*=_file_, *--file*=_file_::
Save output to _file_. Default is stdout.
*-T*=_start_,_end_, *--time-span*=_start_,_end_::
Only export entries between and including _start_ and _end_. _start_ and _end_
are date and time representations according to ISO 8601
(YYYY-MM-DDThh:mm:ss). Time zones are ignored.
Example: `--time-span=2019-01-01,2019-02-10T12:30`.
*-s*=_expression_, *--search-tags*=_expression_::
Search in tags. Format: _tag1 AND tag2 OR tag3_. See _SEARCH EXPRESSIONS_. Case
insensitive.
*-S*=_expression_, *--search-all*=_expression_::
Search in tags, title, description and full text. See _SEARCH EXPRESSIONS_. Case
insensitive.
*-r*, *--regex*::
Use regular expressions for search, case insensitive. With *--search-tags*,
every tag is enclosed by _^_ and _$_.
*-N*, *--no-archive*::
Do not archive URI.
*-d*=_URI_, *--delete*=_URI_::
Remove all entries with this URI from the database.
*-h*, *--help*::
Show help message.
*-V*, *--version*::
Print version, copyright and license.
== EXAMPLES
.Save a thing into the database, with tags.
====
[source,shell]
----
remwharead -t=tag1,tag2 https://example.com/article.html
----
====
.Export all things between and including 2019-04-01 and 2019-05-31 to a file.
====
[source,shell]
----
remwharead -e=asciidoc -f=out.adoc -T=2019-04-01,2019-05-31
----
====
.Export all things to an HTML file.
====
[source,shell]
----
remwharead -e=asciidoc | asciidoctor --backend=html5 --out-file=out.html -
----
====
.Export all things about GRUB the boot-loader, but nothing about caterpillars.
====
[source,shell]
----
remwharead -e=csv -s="grub AND boot"
----
====
.Output all articles by Jan Müller, consider different spellings.
====
[source,shell]
----
remwharead -e=simple -S='Jan[\s]+M(ü|ue?)ller' -r
----
====
.Export all things from the last week to an RSS feed.
====
[source,shell]
----
remwharead -e=rss -T=$(date -d "-1 week" -I),$(date -Iminutes) | sed 's||https://example.com/|' > /var/www/feed.rss
----
====
.Remove all entries that are tagged with mountain
====
[source,shell]
----
OLDIFS=${IFS}
IFS=$'\n'
for uri in $(remwharead -e link -s mountain); do
remwharead -d "${uri}"
done
IFS=${OLDIFS}
----
====
=== Display database
*remwharead* does not provide an interface to display the database. However, you
can periodically generate an HTML file with cron and display it in the browser.
.Line from crontab: Generate an HTML file of the last 6 months every 30 minutes.
====
[source,crontab]
----
*/30 * * * * remwharead -e=asciidoc -T=$(date -d "-6 months" -I),$(date -Iminutes) | asciidoctor --backend=html5 --out-file=${HOME}/remwharead.html -
----
====
== FORMATS
=== csv
CSV is short for comma separated values. All fields are quoted and delimited by
commas. Line breaks in the full text are converted to "\n". Our CSV
implementation follows RFC 4180 and the full MIME media type is
`text/csv;charset=utf-8;header=present`.
=== asciidoc
AsciiDoc is a markup language that can be read as plain text or converted to
HTML, PDF and many other formats.
=== bookmarks
The https://msdn.microsoft.com/en-us/library/aa753582(VS.85).aspx[Netscape
Bookmark file format] is a format for exporting and importing bookmarks that is
understood by most browsers.
=== simple
Simple, human readable, list. Outputs date, title and URI.
=== json
Export as JSON array. See https://tools.ietf.org/html/rfc8259[RFC 8259]. Each
object contains the members _uri_, _archive_uri_, _datetime_, _tags_ (array),
_title_, _description_ and _fulltext_.
=== rss
Export as http://www.rssboard.org/rss-specification[RSS] feed. Because the URL
of the feed is unknown to *remwharead*, the generated feed is slightly out of
specification (the element _link_ in _channel_ is empty).
=== link
Export as a plain list of links, separated by newlines.
=== rofi
Export title, tags and URL for consumption by rofi. See the `scripts/` directory
on https://schlomp.space/tastytea/remwharead for an example.
== SEARCH EXPRESSIONS
A search expression is either a single term, or several terms separated by _AND_
or _OR_. _AND_ takes precedence. The expression _Mountain AND Big OR Vegetable_
finds all things that have either Mountain and Big, or Vegetable in them. You
can use _||_ instead of _OR_ and _&&_ instead of _AND_. Note that
*--search-tags* only matches whole tags, Pill does not match Pillow.
== PROTOCOL SUPPORT
Currently only HTTP and HTTPS are supported.
== PROXY SUPPORT
Since *remwharead* is built on libcurl, it respects the same proxy environment
variables. See *curl*(1), section _ENVIRONMENT_.
Example: http_proxy="http://localhost:3128/"
== FILES
* *Database*: `${XDG_DATA_HOME}/remwharead/database.sqlite`
`${XDG_DATA_HOME}` is usually `~/.local/share`.
== ERROR CODES
[options="header",cols=">,<"]
|====================================================
| Code | Explanation
| 1 | Missing options / Argument not understood.
| 2 | File / database could not be opened.
| 3 | Could not fetch URI.
|====================================================
== SEE ALSO
*crontab*(1), *crontab*(5), *curl*(1)
== REPORTING BUGS
Bugtracker: https://schlomp.space/tastytea/remwharead/issues
E-mail: tastytea@tastytea.de