Bring a pot of salted water to a boil. Add the red potatoes and cook until their tender but still firm (about 15 minutes). Drain potatoes and allow to cool in the refrigerator.
Place eggs in a saucepan, add water until the eggs are covered. Bring to a boil and remove from heat.
Cover and let eggs sit for 10-12 minutes. Remove from heat and plunge eggs into cold water. When eggs are cooled peel the shells off and chop.
Cook bacon until crisp by your method of choice. Place on paper towels to drain, cool and crumble it into bits.
Chop the cooled potatoes, leaving the skin on. Place chopped potatoes in a large bowl. Add eggs, bacon, onion, and celery. Add mayonnaise, salt, and pepper. Mix well and refrigerate for 1 hour before serving.