Although in practice there is rather little confusion in calling certain programming language implementations "compilers," and others "interpreters," if we look at implementations in detail, compiling and interpreting interpenetrate one another. For example, most interpreters condense the text of a source program into a more convenient data structure representing the same information. This step is really a primitive sort of compiling. On the other hand, essentially all compilers contain substantial procedures that are called at various points during program execution (these are called "run-time support"). The running of these procedures is essentially a matter of interpreting part of the behavior of the source program, instead of compiling it.
There seems to be no useful objective mathematical distinction between compilers and interpreters. Pushing the ideas above to extremes, one may always start with a compiler, and "interpret" a program by compiling it and then running it. Conversely, given an interpreter, one may "compile" code consisting of the interpreter itself, followed by inlined data containing the program that was given to be compiled. There seems to be no objective mathematical way to rule out these two tricks, but they clearly violate the spirit and void the utility of the distinction between compilers and interpreters.