Ceph Random Stuff¶

Setting Number of Placement Groups¶

The number of placement groups is calculated the following way:

            (OSDs * 100)
Total PGs = ------------
             Replicas

The number then is rounded up to the next power of two while honoring the magic number of 100 to 200 Target PGs.

             Total PGs
Target PGs = ---------
               OSDs

Example:

• We have 72 OSDs.
• We use Replica = 3 by default.
• The default number of Total PGs calculates to ( (72 * 100) / 3) ) = 2400
• 2400 rounded up would be 4096.
• Total PGs = 4096 gives Target PGs = 56
• Target PGs = 56 is too low, so we use Total PGs = 8192 which gives Target PGs = 113

ceph osd pool set rbd pg_num 8192

Setting Number of Placement Groups for Placement¶

By default, the number of placement grups for placement is set to the same value as the number of placement groups.

ceph osd pool set rbd pgp_num 8192

Increase Ceph MDS Cache¶

# /etc/ceph/ceph.conf

[mds]
# The number of inodes to cache, default 100000
mds_cache_size = 30000000

Increase CephFS Max Files per Directory¶

# /etc/ceph/ceph.conf

[mds]
# hard limit on the size of directory fragments, default 100000
mds_bal_fragment_size_max = 10000000

Adjust recovery settings¶

# /etc/ceph/ceph.conf

osd_max_backfills = 1
osd_recovery_max_active = 1
osd_scrub_during_recovery = false
osd_recovery_op_priority = 1

# online changes
ceph tell osd.* injectargs '--osd_max_backfills=9'

Create Ceph Pool¶

Command to create a new pool:

ceph osd pool create ${POOL} pg_num

Assign application to Pool¶

ceph osd pool application enable ${POOL} rbd
ceph osd pool application enable ${POOL} cephfs