Issue of balance in robotics is best represented by the balancing act of the inverted pendulums. With their unstable and nonlinear behaviour, Inverted Pendulum systems are quite popular systems in which various control design methods are applied and performance comparisons are carried out. In this study, in order to control the pendulum angle and car position of the inverted pendulum system, a fractional order PID controller is designed. Gains of the designed controller are optimized by Particle Swarm Optimization method. On the other hand, for the aim of comparison of performance, conventional PID controllers are used for controllers. The optimum values of gains for PID controllers are also found by same optimization algorithm. The simulation results of both controllers are compared for the inverted pendulum.