When developers learn that modifying Drupal's core code is a no-no, they often have a panic moment. "How, then will I bend Drupal to do my will?," they ask. Easy: by writing a module. The first part of writing a module is writing a .info file, where you describe your module to Drupal. Here's an example from the Forum Module:
; $Id: forum.info,v 1.6 2007/06/08 05:50:54 dries Exp $
name = Forum
description = Enables threaded discussions about general topics.
dependencies = taxonomy
dependencies = comment
package = Core - optional
core = 6.x
This gives Drupalenough information to list the module on the modules administration page, and to tell whether the module is compatible with the version of Drupal being run (in this case, 6.x). Drupal will also make sure the dependent modules are present.
A module may have a .install file containing code that runs when the module is first installed. For example, some database tables may be needed, or some values may need to be initialized in Drupal's persistent variable system.
Finally, the .module file itself contains the code that does whatever it is that your module will do. And that's just about anything. There were 3,430 modules in the repository last time I checked, so it's a good idea to check if the module you're thinking about writing is already written. Drupal Modules is a good place to do that.
New Drupal developers are also often stymied by the question "When does my code run? I put it in a module, but when does the module run?" Answering that question requires understanding of the Inversion of Control design pattern that Drupal uses, often called "hooks" or "callbacks". You name your functions in a certain way, and Drupal will automatically call your code at the appropriate time, depending on how you've named the functions.