{"id":672,"date":"2023-06-17T17:22:54","date_gmt":"2023-06-17T15:22:54","guid":{"rendered":"https:\/\/naupacte.com\/?p=672"},"modified":"2023-06-17T17:22:57","modified_gmt":"2023-06-17T15:22:57","slug":"wikipedia-says-automatic-differentiation-is-algorithmic","status":"publish","type":"post","link":"https:\/\/naupacte.com\/index.php\/en\/2023\/06\/17\/wikipedia-says-automatic-differentiation-is-algorithmic\/","title":{"rendered":"Wikipedia says automatic differentiation is algorithmic!"},"content":{"rendered":"\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>In <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mathematics\">mathematics<\/a> and <a href=\"https:\/\/en.wikipedia.org\/wiki\/Computer_algebra\">computer algebra<\/a>, <strong>automatic differentiation<\/strong> (<strong>auto-differentiation<\/strong>, <strong>autodiff<\/strong>, or <strong>AD<\/strong>), also called <strong>algorithmic differentiation<\/strong>, <strong>computational differentiation<\/strong>,<sup><a href=\"https:\/\/en.wikipedia.org\/wiki\/Automatic_differentiation#cite_note-1\">[1]<\/a><a href=\"https:\/\/en.wikipedia.org\/wiki\/Automatic_differentiation#cite_note-baydin2018automatic-2\">[2]<\/a><\/sup> is a set of techniques to evaluate the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Partial_derivative\">partial derivative<\/a> of a function specified by a computer program.<\/p>\n<cite>https:\/\/en.wikipedia.org\/wiki\/Automatic_differentiation<\/cite><\/blockquote>\n\n\n\n<p>No trace of <em>symbolic<\/em> differentiation, though being in the field of <em>mathematics<\/em> and <em>computer algebra<\/em> (also named symbolic computation in Wikipedia)! Furthermore I don&#8217;t know what an <em>automatic<\/em> differentiation is in <em>mathematics<\/em>: it could only be <em>computer<\/em> or <em>robot<\/em> algebra.<\/p>\n\n\n\n<p>A <em>computer algebra system<\/em> usually deals with mathematical expressions, where the differentiation is said symbolic (though some variants exist in the naming, making conventional distinctions, refer to Wikipedia for <em>symbolic computation vs computer algebra<\/em>.).<\/p>\n\n\n\n<p>If we speak of the differentiation of a program, then it should be said <em>algorithmic<\/em>. Why having chosen such an evocative word, if it can be replaced by the other ones?<\/p>\n\n\n\n<p>However, a program can be seen as a particular <em>procedure-type<\/em> expression, taking as inputs the parameters and the body of the function; and conversely, an expression can be transformed into a procedure according to an evaluation mechanism (cf. Wikipedia, article Computer_algebra, section Expressions, paragraph 2). Then all is confused&#8230;<\/p>\n\n\n\n<p>Nonetheless, to sum up:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>algorithmic differentiation is a formal differentiation operating on the expressions that the elements of a program constitute.<\/li>\n\n\n\n<li>symbolic differentiation is a formal differentiation operating on classical mathematical expressions.<\/li>\n<\/ul>\n\n\n\n<p>The name auto-differentiation (or auto-diff) is a bad idea because <em>auto<\/em> itself means <em>self<\/em> and should not stand for <em>automatic<\/em>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">&#8230; and in no way symbolic nor numerical<\/h2>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Automatic differentiation is distinct from <a href=\"https:\/\/en.wikipedia.org\/wiki\/Symbolic_differentiation\">symbolic differentiation<\/a> and <a href=\"https:\/\/en.wikipedia.org\/wiki\/Numerical_differentiation\">numerical differentiation<\/a><\/p>\n<\/blockquote>\n\n\n\n<p>Symbolic differentiation is an automatic differentiation: otherwise, I don&#8217;t know what we&#8217;re talking about, except for the high school student on his hand-made exercises.<\/p>\n\n\n\n<p>Numerical differentiation is not automatic as a technique in itself but is probably the most easily automated method. And even it becomes, in a somewhat abusive manner, an algorithmic differentiation to the coarsest grain&#8230;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Both methods are defined only by their flaws<\/h2>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p>Symbolic differentiation faces the difficulty of converting a computer program into a single <a href=\"https:\/\/en.wikipedia.org\/wiki\/Mathematical_expression\">mathematical expression<\/a> and can lead to inefficient code. Numerical differentiation (the method of finite differences) can introduce <a href=\"https:\/\/en.wikipedia.org\/wiki\/Round-off_error\">round-off errors<\/a> in the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Discretization\">discretization<\/a> process and cancellation. Both of these classical methods have problems with calculating higher derivatives, where complexity and errors increase. Finally, both of these classical methods are slow at computing partial derivatives of a function with respect to <em>many<\/em> inputs, as is needed for <a href=\"https:\/\/en.wikipedia.org\/wiki\/Gradient_descent\">gradient<\/a>-based <a href=\"https:\/\/en.wikipedia.org\/wiki\/Optimization_(mathematics)\">optimization<\/a> algorithms.<\/p>\n<cite>Idem<\/cite><\/blockquote>\n\n\n\n<p>How do these flaws prevent a method from being automatic? To lead to an inefficient code is to arrive at a code!<\/p>\n\n\n\n<p>The ideas presented should be developed further.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">&#8230; and automatic differentiation solves all these problems (sic)<\/h2>\n\n\n\n<p>It says it all in that title, which is the conclusion of the second paragraph. Neutral style.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">My conclusion: a Wikipedia entry for algorithmic <meta http-equiv=\"content-type\" content=\"text\/html; charset=utf-8\"><\/meta>differentiation<\/h2>\n\n\n\n<p>It is necessary to return the term <em>automatic<\/em> to its usual meaning: <em>automatic<\/em> does not mean <em>algorithmic<\/em>, except abusively in a special context. <em>Algorithmic<\/em> differentiation is an automatic differentiation of code. The other automatic differentiations must be presented and shown how they differ, in particular their advantages and their disadvantages, and reference must be made to a special article for algorithmic differentiation, thus avoiding this double bias: usurped equivalence with one method and relegation of the competing methods.<\/p>\n\n\n\n<p>Wikipedia is also faulty: a search for &#8220;algorithmic differentiation&#8221; or &#8220;computational differentiation&#8221;  lands back to &#8220;Automatic_differentiation&#8221;. It&#8217;s marble-graved equivalent, you&#8217;re told!<\/p>\n\n\n\n<p>Nonetheless, the relevant scheme involving symbolic and algorithmic <meta http-equiv=\"content-type\" content=\"text\/html; charset=utf-8\"><\/meta>differentiation must be used and detailed. <em>See button below<\/em>.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/naupacte.com\/index.php\/en\/numerical-modeling-simulation-actors-2\/numerical-simulation-automatic-differentiation-en\/\">Numerical simulation and automatic differentiation<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>In mathematics and computer algebra, automatic differentiation (auto-differentiation, autodiff, or AD), also called algorithmic differentiation, computational differentiation,[1][2] is a set of techniques to evaluate the partial derivative of a function specified by a computer program. https:\/\/en.wikipedia.org\/wiki\/Automatic_differentiation No trace of symbolic differentiation, though being in the field of mathematics and computer algebra (also named symbolic computation in Wikipedia)! Furthermore I don&#8217;t know what an automatic differentiation is in mathematics: it could only be computer or robot algebra. A computer algebra system usually deals with mathematical expressions, where the differentiation is said symbolic (though some variants exist in the naming, making conventional distinctions, refer to Wikipedia for symbolic computation vs computer algebra.). If we speak of the differentiation of a program, then it should be said algorithmic. Why having chosen such an evocative word, if it can be replaced by the other ones? However, a program can be seen as a particular procedure-type expression, taking as inputs the parameters and the body of the function; and conversely, an expression can be transformed into a procedure according to an evaluation mechanism (cf. Wikipedia, article Computer_algebra, section Expressions, paragraph 2). Then all is confused&#8230; Nonetheless, to sum up: The name auto-differentiation (or auto-diff) is a bad idea because auto itself means self and should not stand for automatic. &#8230; and in no way symbolic nor numerical Automatic differentiation is distinct from symbolic differentiation and numerical differentiation Symbolic differentiation is an automatic differentiation: otherwise, I don&#8217;t know what we&#8217;re talking about, except for the high school student on his hand-made exercises. Numerical differentiation is not automatic as a technique in itself but is probably the most easily automated method. And even it becomes, in a somewhat abusive manner, an algorithmic differentiation to the coarsest grain&#8230; Both methods are defined only by their flaws Symbolic differentiation faces the difficulty of converting a computer program into a single mathematical expression and can lead to inefficient code. Numerical differentiation (the method of finite differences) can introduce round-off errors in the discretization process and cancellation. Both of these classical methods have problems with calculating higher derivatives, where complexity and errors increase. Finally, both of these classical methods are slow at computing partial derivatives of a function with respect to many inputs, as is needed for gradient-based optimization algorithms. Idem How do these flaws prevent a method from being automatic? To lead to an inefficient code is to arrive at a code! The ideas presented should be developed further. &#8230; and automatic differentiation solves all these problems (sic) It says it all in that title, which is the conclusion of the second paragraph. Neutral style. My conclusion: a Wikipedia entry for algorithmic differentiation It is necessary to return the term automatic to its usual meaning: automatic does not mean algorithmic, except abusively in a special context. Algorithmic differentiation is an automatic differentiation of code. The other automatic differentiations must be presented and shown how they differ, in particular their advantages and their disadvantages, and reference must be made to a special article for algorithmic differentiation, thus avoiding this double bias: usurped equivalence with one method and relegation of the competing methods. Wikipedia is also faulty: a search for &#8220;algorithmic differentiation&#8221; or &#8220;computational differentiation&#8221; lands back to &#8220;Automatic_differentiation&#8221;. It&#8217;s marble-graved equivalent, you&#8217;re told! Nonetheless, the relevant scheme involving symbolic and algorithmic differentiation must be used and detailed. See button below.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[43],"tags":[],"class_list":["post-672","post","type-post","status-publish","format-standard","hentry","category-programmation-en"],"_links":{"self":[{"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/posts\/672","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/comments?post=672"}],"version-history":[{"count":1,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/posts\/672\/revisions"}],"predecessor-version":[{"id":673,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/posts\/672\/revisions\/673"}],"wp:attachment":[{"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/media?parent=672"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/categories?post=672"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naupacte.com\/index.php\/wp-json\/wp\/v2\/tags?post=672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}