In this paper, we consider the feature selection problem in unsupervised learning scenarios. Based on Laplacian regularized least squares, we propose two novel feature selection algorithms and  select those features such that the size of the parameter covariance matrix of the regularized regression model is minimized. We use trace and determinant operators to measure the size of the covariance matrix. Efficient computational schemes are also introduced to solve the corresponding optimization problems. Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithms.