Current knowledge acquisition tools lack understanding of what users are doing and how well they are doing, and cannot provide effective assistance in organizing various knowledge authoring tasks. Users have to make up for these shortcomings by keeping track of the status, progress, potential problems and possible courses of actions by themselves. We present a novel extension to existing systems that 1) keeps track of past problem solving episodes and relates them to user entered knowledge, 2) assesses the current status of the knowledge and the problem solving using such relations, and 3) provides assistance to the user based on the assessment. We applied the approach in developing an intelligent assistant for decision making tasks. The resulting interaction shows that the system helps the user understand the progress and guides the knowledge authoring process in terms of making the knowledge more useful, adapting the knowledge to dynamic changes over time, and making the overall problem solving more successful.