Peppers are a warm weather plant.  Flowers will drop from the plant if temperatures are over 85 degrees.  Harvest first peppers promptly to stimulate further fruit production.  Colored peppers generally require an additional 2-4 weeks to fully ripen.