In mathematics and computer algebra, automatic differentiation (auto-differentiation, autodiff, or AD), also called algorithmic differentiation, computational differentiation,[1][2] is a set of techniques to evaluate the partial derivative of a function specified by a computer program.
https://en.wikipedia.org/wiki/Automatic_differentiation
No trace of symbolic differentiation, though being in the field of mathematics and computer algebra (also named symbolic computation in Wikipedia)! Furthermore I don’t know what an automatic differentiation is in mathematics: it could only be computer or robot algebra.
A computer algebra system usually deals with mathematical expressions, where the differentiation is said symbolic (though some variants exist in the naming, making conventional distinctions, refer to Wikipedia for symbolic computation vs computer algebra.).
If we speak of the differentiation of a program, then it should be said algorithmic. Why having chosen such an evocative word, if it can be replaced by the other ones?
However, a program can be seen as a particular procedure-type expression, taking as inputs the parameters and the body of the function; and conversely, an expression can be transformed into a procedure according to an evaluation mechanism (cf. Wikipedia, article Computer_algebra, section Expressions, paragraph 2). Then all is confused…
Nonetheless, to sum up:
- algorithmic differentiation is a formal differentiation operating on the expressions that the elements of a program constitute.
- symbolic differentiation is a formal differentiation operating on classical mathematical expressions.
The name auto-differentiation (or auto-diff) is a bad idea because auto itself means self and should not stand for automatic.
… and in no way symbolic nor numerical
Automatic differentiation is distinct from symbolic differentiation and numerical differentiation
Symbolic differentiation is an automatic differentiation: otherwise, I don’t know what we’re talking about, except for the high school student on his hand-made exercises.
Numerical differentiation is not automatic as a technique in itself but is probably the most easily automated method. And even it becomes, in a somewhat abusive manner, an algorithmic differentiation to the coarsest grain…
Both methods are defined only by their flaws
Symbolic differentiation faces the difficulty of converting a computer program into a single mathematical expression and can lead to inefficient code. Numerical differentiation (the method of finite differences) can introduce round-off errors in the discretization process and cancellation. Both of these classical methods have problems with calculating higher derivatives, where complexity and errors increase. Finally, both of these classical methods are slow at computing partial derivatives of a function with respect to many inputs, as is needed for gradient-based optimization algorithms.
Idem
How do these flaws prevent a method from being automatic? To lead to an inefficient code is to arrive at a code!
The ideas presented should be developed further.
… and automatic differentiation solves all these problems (sic)
It says it all in that title, which is the conclusion of the second paragraph. Neutral style.
My conclusion: a Wikipedia entry for algorithmic differentiation
It is necessary to return the term automatic to its usual meaning: automatic does not mean algorithmic, except abusively in a special context. Algorithmic differentiation is an automatic differentiation of code. The other automatic differentiations must be presented and shown how they differ, in particular their advantages and their disadvantages, and reference must be made to a special article for algorithmic differentiation, thus avoiding this double bias: usurped equivalence with one method and relegation of the competing methods.
Wikipedia is also faulty: a search for “algorithmic differentiation” or “computational differentiation” lands back to “Automatic_differentiation”. It’s marble-graved equivalent, you’re told!
Nonetheless, the relevant scheme involving symbolic and algorithmic differentiation must be used and detailed. See button below.