What parts outside the brain are you referencing? The only way we can sense anything going on outside our brain is if we have a sensory apparatus transducing information from it and sending information up to our brain, and our experience of the smell of chocolate is no different - we have our olfactory nerve taking in the smell, our pyriform cortex processing it, and supplemental areas such as the hippocampus, amygdala, and association areas integrating it with our memories and emotions.

If you're talking about memory, the current consensus is that it's distributed over the entire brain and is manifested in synapse changes, last I heard (motor learning especially, as it goes from requiring a lot of motor cortical involvement to being controlled by the spinal cord in many instances - cats can walk with support even after you transect their brain just above the brainstem). But it is a brain thing.

From Wikipedia:

Brain areas involved in the neuroanatomy of memory such as the hippocampus, the amygdala, the striatum, or the mammillary bodies are thought to be involved in specific types of memory. For example, the hippocampus is believed to be involved in spatial learning and declarative learning, while the amygdala is thought to be involved in emotional memory.[32] Damage to certain areas in patients and animal models and subsequent memory deficits is a primary source of information. However, rather than implicating a specific area, it could be that damage to adjacent areas, or to a pathway traveling through the area is actually responsible for the observed deficit. Further, it is not sufficient to describe memory, and its counterpart, learning, as solely dependent on specific brain regions. Learning and memory are attributed to changes in neuronal synapses, thought to be mediated by long-term potentiation and long-term depression.

In general, the more emotionally charged an event or experience is, the better it is remembered; this phenomenon is known as the memory enhancement effect. Patients with amygdala damage, however, do not show a memory enhancement effect.[33][34]

Hebb distinguished between short-term and long-term memory. He postulated that any memory that stayed in short-term storage for a long enough time would be consolidated into a long-term memory. Later research showed this to be false. Research has shown that direct injections of cortisol or epinephrine help the storage of recent experiences. This is also true for stimulation of the amygdala. This proves that excitement enhances memory by the stimulation of hormones that affect the amygdala. Excessive or prolonged stress (with prolonged cortisol) may hurt memory storage. Patients with amygdalar damage are no more likely to remember emotionally charged words than nonemotionally charged ones. The hippocampus is important for explicit memory. The hippocampus is also important for memory consolidation. The hippocampus receives input from different parts of the cortex and sends its output out to different parts of the brain also. The input comes from secondary and tertiary sensory areas that have processed the information a lot already. Hippocampal damage may also cause memory loss and problems with memory storage.