Given $k\ge 2$ and two $k$-graphs ($k$-uniform hypergraphs) $F$ and $H$, an $F$-factor in $H$ is a set of vertex disjoint copies of $F$ that together covers the vertex set of $H$. Lenz and Mubayi [J. Combin. Theory Ser. B, 2016] studied the $F$-factor problem in quasi-random $k$-graphs with minimum degree $\Omega(n^{k-1})$. They posed the problem of characterizing the $k$-graphs $F$ such that every sufficiently large quasi-random $k$-graph with constant edge density and minimum degree $\Omega(n^{k-1})$ contains an $F$-factor, and in particular, they showed that all linear $k$-graphs satisfy this property. In this paper we prove a general theorem on $F$-factors which reduces the $F$-factor problem of Lenz and Mubayi to a natural sub-problem, that is, the $F$-cover problem. By using this result, we answer the question of Lenz and Mubayi for those $F$ which are $k$-partite $k$-graphs, and for all 3-graphs $F$, separately. Our characterization result on 3-graphs is motivated by the recent work of Reiher, R\"odl and Schacht [J. Lond. Math. Soc., 2018] that classifies the 3-graphs with vanishing Tur\'an density in quasi-random $k$-graphs.