Skip to content

dovecot: add tmpfs_index chatmail.ini parameter for storing index files in /dev/shm#757

Open
missytake wants to merge 4 commits intomainfrom
tmpfs-index
Open

dovecot: add tmpfs_index chatmail.ini parameter for storing index files in /dev/shm#757
missytake wants to merge 4 commits intomainfrom
tmpfs-index

Conversation

@missytake
Copy link
Contributor

fix #644

@missytake
Copy link
Contributor Author

Maybe we can just use https://linuxvox.com/blog/devshm-in-linux/ instead of shipping our own tmpfs.

@missytake missytake changed the title dovecot: add tmpfs for storing dovecot.index.cache in RAM WIP: dovecot: add tmpfs for storing dovecot.index.cache in RAM Dec 8, 2025
Copy link
Contributor

@hpk42 hpk42 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dovecot allows to set a separate index directory: https://doc.dovecot.org/2.3/configuration_manual/mail_location/#index-files which is preferable to creating symlinks

how is dovecot handling it when the (tmp) file system gets full?

@missytake missytake changed the title WIP: dovecot: add tmpfs for storing dovecot.index.cache in RAM WIP: dovecot: add tmpfs for storing dovecot INDEX files in /dev/shm Feb 3, 2026
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 11:07 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 11:07 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 11:37 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 11:37 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 11:57 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 11:57 — with GitHub Actions Inactive
@missytake missytake changed the title WIP: dovecot: add tmpfs for storing dovecot INDEX files in /dev/shm dovecot: add tmpfs_index chatmail.ini parameter for storing index files in /dev/shm Feb 3, 2026
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 12:44 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 12:44 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 12:49 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 12:49 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging2.testrun.org February 3, 2026 14:44 — with GitHub Actions Inactive
@missytake missytake temporarily deployed to staging-ipv4.testrun.org February 3, 2026 14:44 — with GitHub Actions Inactive
# but it's not clear how to disable them completely.
# According to https://doc.dovecot.org/2.3/settings/advanced/#core_setting-mail_cache_max_size
# if the cache file becomes larger than the specified size, it is truncated by dovecot
mail_cache_max_size = 500K
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing this limit will lead to no limit when we dont use the tmpfs option I presume?

@j4n
Copy link
Contributor

j4n commented Feb 24, 2026

how is dovecot handling it when the (tmp) file system gets full?

This is a good question, along with the usual combined size of the caches on a large server.
FWIW, on docker shm defaults to 64 MiB, so that would need an override parameter exposed.

self.remove_file(f"{mbox.basedir}/maildirsize")
for file in mbox.extrafiles:
if "dovecot.index" in file.path.split("/")[-1] and file.size > 500 * 1024:
self.remove_file(file.path)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This matches broadly and according to my brief research triggers a full index rebuild where the old max_size flag just pruned the cache, I wonder if this won't mean a lot of I/O churn.

# Certificate renewal is your responsibility; changed files are
# picked up automatically by all relay services.
# tls_external_cert_and_key = /path/to/fullchain.pem /path/to/privkey.pem
# store index files in tmpfs (good for disk size and I/O, bad for ram)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be more informative I think. What are the performance implications / motivations to enable this? How big should your ram / tmpfs wrt. to number of chatmail accounts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Restore in-memory index

3 participants