Review the probate laws of your jurisdiction to determine whether you have standing to challenge the executor. Since executor challenges delay probate and deplete the estate -- it pays for executor and attorney fees -- many state statutes only allow heirs to object to the qualifications or actions of an executor. A close personal relationship with the deceased is usually not sufficient to confer standing.
Investigate the character and background of the executor, if you intend to challenge on capacity. Recognize that the courts presume an adult to have the capacity for sound reasoning; allegations of mental incapacity must overcome that presumption. Look for evidence of a severe mental, physical or moral limitation making it impossible for the executor to serve. Proof of criminal behavior leading to jail time persuades most courts of moral incapacity.
View your evidence of improper behavior with clear eyes. Although executors owe heirs a fiduciary duty -- the highest duty imposed by law -- only serious infractions constitute grounds for disqualification. Courts will not disqualify an executor because an heir disagrees with her on an investment strategy. Nor will courts listen with favor to heirs' complaints that the executor's progress is too slow or inefficient. Instead, gather evidence of embezzlement, lies to the court, self-dealing or other malfeasance.
Hire a lawyer to file a timely objection. Alternatively, research the probate procedures in your jurisdiction and file your own. Rules governing the timing of most objections vary among jurisdictions. When the court receives an objection, it sets a trial date and notifies other heirs and the executor. Prepare your case, appear on the correct date and present your evidence to the court. Convince the court of the veracity of your allegations. The judge will either rule from the bench -- tell you her decision just after final argument -- or else take the matter under advisement and mail you a written decision later.