We develop an exact method to calculate thermal Casimir forces between inclusions of arbitrary shapes and separation, embedded in a fluid membrane whose fluctuations are governed by the combined action of surface tension, bending modulus, and Gaussian rigidity. Each object's shape and mechanical properties enter only through a characteristic matrix, a static analog of the scattering matrix. We calculate the Casimir interaction between two elastic disks embedded in a membrane. In particular, we find that at short separations the interaction is strong and independent of surface tension.