6.3 KiB

Raw Permalink Blame History

remwharead(1) Manual Page

NAME

remwharead - Saves URIs of things you want to remember in a database

SYNOPSIS

remwharead [-t=tags] [-N] URI

remwharead -e=format [-f=file] [-T=start,end] [[-s|-S]=expression] [-r]

remwharead [-d=URI]

DESCRIPTION

remwharead saves URIs of things you want to remember in a database along with an URI to the archived version, the current date and time, title, description, the full text of the page and optional tags.

The database can be filtered by time, tags and full text and exported to CSV, AsciiDoc, a bookmarks file, JSON, RSS, a list of hyperlinks or a rofi-compatible list.

Archiving is done using the Wayback machine from the Internet Archive.

OPTIONS

-t=tags, --tags=tags: Add tags to URI, delimited by commas.
-e=format, --export=format: Export to format. Possible values are csv, asciidoc, bookmarks, simple, json, rss, link or rofi. See FORMATS.
-f=file, --file=file: Save output to file. Default is stdout.
-T=start,end, --time-span=start,end: Only export entries between and including start and end. start and end are date and time representations according to ISO 8601 (YYYY-MM-DDThh:mm:ss). Time zones are ignored. Example: --time-span=2019-01-01,2019-02-10T12:30.
-s=expression, --search-tags=expression: Search in tags. Format: tag1 AND tag2 OR tag3. See SEARCH EXPRESSIONS. Case insensitive.
-S=expression, --search-all=expression: Search in tags, title, description and full text. See SEARCH EXPRESSIONS. Case insensitive.
-r, --regex: Use regular expressions for search, case insensitive. With --search-tags, every tag is enclosed by ^ and $.
-N, --no-archive: Do not archive URI.
-d=URI, --delete=URI: Remove all entries with this URI from the database.
-h, --help: Show help message.
-V, --version: Print version, copyright and license.

EXAMPLES

Example 1. Save a thing into the database, with tags.

remwharead -t=tag1,tag2 https://example.com/article.html

Example 2. Export all things between and including 2019-04-01 and 2019-05-31 to a file.

remwharead -e=asciidoc -f=out.adoc -T=2019-04-01,2019-05-31

Example 3. Export all things to an HTML file.

remwharead -e=asciidoc | asciidoctor --backend=html5 --out-file=out.html -

Example 4. Export all things about GRUB the boot-loader, but nothing about caterpillars.

remwharead -e=csv -s="grub AND boot"

Example 5. Output all articles by Jan Müller, consider different spellings.

remwharead -e=simple -S='Jan[\s]+M(ü|ue?)ller' -r

Example 6. Export all things from the last week to an RSS feed.

remwharead -e=rss -T=$(date -d "-1 week" -I),$(date -Iminutes) | sed 's|<link/>|<link>https://example.com/</link>|' > /var/www/feed.rss

Example 7. Remove all entries that are tagged with mountain

OLDIFS=${IFS}
IFS=$'\n'

for uri in $(remwharead -e link -s mountain); do
    remwharead -d "${uri}"
done

IFS=${OLDIFS}

Display database

remwharead does not provide an interface to display the database. However, you can periodically generate an HTML file with cron and display it in the browser.

Example 8. Line from crontab: Generate an HTML file of the last 6 months every 30 minutes.

*/30 * * * * remwharead -e=asciidoc -T=$(date -d "-6 months" -I),$(date -Iminutes) | asciidoctor --backend=html5 --out-file=${HOME}/remwharead.html -

FORMATS

csv

CSV is short for comma separated values. All fields are quoted and delimited by commas. Line breaks in the full text are converted to "\n". Our CSV implementation follows RFC 4180 and the full MIME media type is text/csv;charset=utf-8;header=present.

asciidoc

AsciiDoc is a markup language that can be read as plain text or converted to HTML, PDF and many other formats.

bookmarks

The Netscape Bookmark file format is a format for exporting and importing bookmarks that is understood by most browsers.

simple

Simple, human readable, list. Outputs date, title and URI.

json

Export as JSON array. See RFC 8259. Each object contains the members uri, archive_uri, datetime, tags (array), title, description and fulltext.

rss

Export as RSS feed. Because the URL of the feed is unknown to remwharead, the generated feed is slightly out of specification (the element link in channel is empty).

link

Export as a plain list of links, separated by newlines.

rofi

Export title, tags and URL for consumption by rofi. See the scripts/ directory on https://schlomp.space/tastytea/remwharead for an example.

SEARCH EXPRESSIONS

A search expression is either a single term, or several terms separated by AND or OR. AND takes precedence. The expression Mountain AND Big OR Vegetable finds all things that have either Mountain and Big, or Vegetable in them. You can use || instead of OR and && instead of AND. Note that --search-tags only matches whole tags, Pill does not match Pillow.

PROTOCOL SUPPORT

Currently only HTTP and HTTPS are supported.

PROXY SUPPORT

Since remwharead is built on libcurl, it respects the same proxy environment variables. See curl(1), section ENVIRONMENT.

Example: http_proxy="http://localhost:3128/"

FILES

Database: ${XDG_DATA_HOME}/remwharead/database.sqlite

${XDG_DATA_HOME} is usually ~/.local/share.

ERROR CODES

Code	Explanation
1	Missing options / Argument not understood.
2	File / database could not be opened.
3	Could not fetch URI.

REPORTING BUGS

Bugtracker: https://schlomp.space/tastytea/remwharead/issues

E-mail: tastytea@tastytea.de

6.3 KiB Raw Permalink Blame History