This is a project that emphasis on developing a robust, accurate and efficient speech recognition system for phonetic language. We started the project with aim of developing a Continuous Speech Recognition System in Malayalam language. To start with we use Carnegie Mellon University's Sphinx engine as our base.