Operator learning, learning the mapping between function spaces, has been attracted as an alternative approach to traditional numerical methods to solve partial differential equations. In this paper, we propose to represent the discretized system as a set-valued data without a prior structure and construct the permutation-symmetric model, called mesh-independent neural operator (MINO), to provide proper treatments of input functions and query coordinates of the solution function. Our models pre-trained with a benchmark dataset of operator learning are evaluated by downstream tasks to demonstrate the generalization abilities to varying discretization formats of the system, which are natural characteristics of the continuous solution of the PDEs.