Write
\(Z = X \cup_A B\) and let
\(\varphi \colon X \to Z\) and
\(\psi \colon B \to Z\) be the canonical maps, so that
\(\varphi|_A = \psi \circ f\text{.}\) Suppose given
\(g \colon Z \to Y\) and a homotopy
\(G \colon B \times I \to Y\) with
\(G_0 = g \circ \psi\text{.}\) Then
\(g \circ \varphi \colon X \to Y\) is a map whose restriction to
\(A\) is
\(g \circ \psi \circ f = G_0 \circ f\text{,}\) so
\(G \circ (f \times \mathrm{id}_I) \colon A \times I \to Y\) is a homotopy starting at
\((g \circ \varphi)|_A\text{.}\) By the HEP for
\((X, A)\) it extends to a homotopy
\(\widetilde{G} \colon X \times I \to Y\) starting at
\(g \circ \varphi\text{.}\) Now
\(\widetilde{G}\) and
\(G\) agree on
\(A \times I\) (after composing with
\(f \times \mathrm{id}_I\)), so they glue to a map
\(Z \times I \to Y\) by the universal property of the pushout
\(Z \times I = (X \times I) \cup_{A \times I} (B \times I)\text{.}\) This is the required extension: it restricts to
\(G\) on
\(B \times I\) and to
\(g\) at time
\(0\text{.}\)