Chimneys rely on pressure to carry the smoke out of your home. The smoke can sometimes be carried back into your home if you haven’t allowed the chimney to heat up properly. To avoid the smoke from coming back into your home you should first light up some newspaper or kindling, which allows the chimney to slowly heat up. As the temperature in the chimney gradually heats up, you can begin to add your logs.
If you still experience a down draft when your fireplace is in use, turn off your exhaust fans. They naturally increase negative pressure by pulling air out of the house, which can cause a down draft. If your damper is warped, you might need to adjust it slightly to get the right airflow. This helps with negative pressure by providing potentially another path for air to enter the house instead of rushing down the chimney.
Ironically, if two or more windows are open, try closing them. A breeze coming through these and crossing the room could also draw a vacuum in the house.
If you live in an old home, it is possible that your chimney might be too short. Short chimneys cannot vent properly, it should at least be three to four feet tall. Soot and burned debris buildup can narrow passageways for the smoke to escape through your flue.