Vector Operations — A Complete Reference

1. What is a Vector?

A vector carries two ideas at once. Geometrically, a vector is an arrow with a definite magnitude (length) and direction; its starting point does not matter, so “three steps east and four steps north” is the same vector wherever you begin. Algebraically, the same object is an ordered tuple of numbers — (3, 4) in 2D, (1, −2, 5) in 3D — where each entry is a coordinate along a chosen axis.

These views are two sides of one idea. The arrow form gives intuition; the tuple form gives computability. The leap from “arrows in space” to “elements of ℝⁿ” is what lets the same vocabulary describe a 2D screen position, a 3D physics force, and a 768-dimensional word embedding inside a language model.

A small distinction matters in linear algebra. A row vector writes its entries horizontally, [3, 4]; a column vector stacks them vertically. Both contain identical numbers yet behave differently under matrix multiplication: a column vector multiplies on the right of a matrix (Ax), a row vector on the left (x^TA). Most modern textbooks and most software default to column vectors, and we follow that convention.

벡터는 "크기와 방향을 가진 화살표"이자 "순서 있는 수의 묶음"이라는 두 얼굴을 동시에 갖습니다. 같은 객체를 기하적으로 보면 직관을, 대수적으로 보면 계산력을 얻으며, 이 둘의 연결이 2D 좌표·3D 힘·고차원 임베딩을 같은 언어로 다루게 해 줍니다.

2. Notation & Special Vectors

Mathematical writing represents vectors in interchangeable ways: textbooks use boldface (v, a, b), handwritten work uses an arrow on top (v⃗, a⃗, b⃗), and physics mixes the two. Entries take subscripts: v = (v₁, …, v_n). Be consistent inside a document — silently switching styles is a frequent source of confusion.

Several special vectors appear everywhere. The zero vector 0 has every entry equal to 0; it is the additive identity (v + 0 = v) and the only vector whose direction is undefined. A unit vector has magnitude 1; the hat v̂ signals “normalized.” The standard basis vectors in ℝ³ are e₁ = (1, 0, 0), e₂ = (0, 1, 0), e₃ = (0, 0, 1), also written î, ĵ, k̂; every v ∈ ℝ³ decomposes uniquely as v = v₁e₁ + v₂e₂ + v₃e₃, which is exactly why coordinates work the way they do.

A subtler distinction is between position vectors and displacement vectors. A position vector points from the origin to a fixed location. A displacement represents change: “move from P to Q,” computed as Q − P. Subtracting two positions is a displacement; adding a displacement to a position gives a new position; but adding two positions has no geometric meaning.

벡터는 굵은 글씨(**v**)나 화살표(v⃗)로 적고, 영벡터 **0**·단위벡터 v̂·표준 기저 e₁, e₂, e₃가 특수 벡터로 자주 등장합니다. 위치 벡터(원점으로부터의 좌표)와 변위 벡터(두 점의 차이)는 다른 개념이며, 둘을 혼동하면 물리·그래픽스 계산이 어긋납니다.

3. Vector Addition & Scalar Multiplication

Two vectors of the same dimension are added component by component: (a₁, …, a_n) + (b₁, …, b_n) = (a₁ + b₁, …, a_n + b_n). You cannot add a vector in ℝ² to one in ℝ³ — dimensions must match, and numerical libraries throw “shape mismatch” errors when they do not.

Geometrically, addition obeys two equivalent rules. The head-to-tail rule places the tail of b at the head of a; the sum is the arrow drawn from the original tail to the new head. The parallelogram rule places a and b with a common tail and completes the parallelogram; the diagonal from the shared tail is the sum. Both rules agree because addition is commutative (a + b = b + a) and associative ((a + b) + c = a + (b + c)).

Scalar multiplication stretches or shrinks a vector. Multiplying v by a real number k yields kv = (kv₁, …, kv_n). The geometric effect depends on the sign of k: a positive k stretches v in the same direction (k = 2 doubles its length), a negative k flips it across the origin, and a fractional k with 0 < |k| < 1 shrinks it.

A linear combination of v₁, …, v_m is any expression c₁v₁ + … + c_mv_m with scalar coefficients. The set of all such combinations is called their span: one nonzero vector spans a line, two non-parallel vectors in ℝ³ span a plane, three vectors in general position span all of ℝ³. Span is the foundational idea behind dimension, basis, and almost every theorem in linear algebra.

벡터 덧셈은 같은 차원끼리 성분별 합으로 정의되며 기하적으로는 머리-꼬리(혹은 평행사변형) 규칙을 따릅니다. 스칼라 곱은 길이를 늘이거나(양수) 뒤집고(음수) 줄이는(소수) 효과를 가지며, c₁v₁ + … + c_mv_m 형태의 일차결합과 그 전체 집합인 생성공간(span)은 선형대수의 출발점입니다.

4. Dot Product (Inner Product)

The dot product combines two vectors of the same dimension into a single number. The algebraic definition is the sum of pairwise products:

a · b = Σ_i=1..n a_ib_i = a₁b₁ + a₂b₂ + … + a_nb_n

There is an equivalent geometric definition that ties this scalar to angle:

a · b = ||a|| · ||b|| · cos θ

where θ is the angle between the two vectors. The definitions agree exactly, and together they reveal why the dot product is so useful: it simultaneously measures how long each vector is and how aligned the two are. When the vectors point the same way, cos θ = 1 and the product is maximal; when perpendicular, cos θ = 0 and the product vanishes; when opposite, cos θ = −1.

Worked example. Let a = (1, 2, 3) and b = (4, −5, 6). Then a · b = 1·4 + 2·(−5) + 3·6 = 4 − 10 + 18 = 12. The positive value means the angle between a and b is acute; for the exact angle, divide by the product of the magnitudes (see section 8).

The dot product has a direct physical reading: in mechanics, the work done by a constant force F over displacement d is W = F · d — only the component of force along the motion contributes, which is exactly what the dot product extracts. In computer graphics, the brightness of a surface under a directional light is N · L (surface normal dotted with light direction), clamped to be non-negative. In information retrieval and machine learning, dot products of TF-IDF or embedding vectors quantify how related two documents or sentences are.

내적은 두 벡터를 하나의 수로 묶는 연산으로, 대수적으로는 a·b = Σa_ib_i, 기하적으로는 a·b = ||a||·||b||·cos θ로 같은 값을 줍니다. 부호는 두 벡터의 방향 일치 정도를 나타내며, 물리의 일(W = F·d), 그래픽스의 조명 계산(N·L), 머신러닝의 유사도(코사인 유사도) 등에 두루 쓰입니다.

5. Cross Product

The cross product is a uniquely 3-dimensional construction (also definable in 7D, but that case never appears in practice). For a = (a₁, a₂, a₃) and b = (b₁, b₂, b₃) in ℝ³, the cross product a × b is another vector, perpendicular to both, given by:

a × b = (a₂b₃ − a₃b₂, a₃b₁ − a₁b₃, a₁b₂ − a₂b₁)

The same formula is usually memorized as the symbolic expansion of a 3×3 determinant whose top row is î, ĵ, k̂ and whose lower two rows are the components of a and b.

The direction of a × b is fixed by the right-hand rule: curl the fingers of your right hand from a toward b, and your thumb points along a × b. The cross product is therefore anticommutative — a × b = −b × a — and it is also not associative: in general (a × b) × c ≠ a × (b × c).

The magnitude has a concrete reading: ||a × b|| = ||a|| · ||b|| · sin θ, which equals the area of the parallelogram spanned by a and b. When a and b are parallel, sin θ = 0 and the cross product vanishes, making it an excellent parallelism test in 3D.

Worked example. Let a = (1, 0, 0) and b = (0, 1, 0). Then a × b = (0·0 − 0·1, 0·0 − 1·0, 1·1 − 0·0) = (0, 0, 1). The product points along the positive z-axis — exactly what the right-hand rule predicts when curling from x toward y — with magnitude 1, matching the unit square.

외적은 3차원에서만 의미가 있는 연산으로, 두 벡터에 동시에 수직한 새 벡터를 반환합니다. 방향은 오른손 법칙, 크기는 두 벡터가 만드는 평행사변형의 넓이(||a||·||b||·sin θ)에 해당하며, a × b = −b × a라는 반교환 법칙이 성립합니다. 평행한 두 벡터의 외적은 0벡터입니다.

6. Magnitude & Unit Vectors

The magnitude (or Euclidean norm) of v = (v₁, …, v_n) is its length:

||v|| = √(v₁² + v₂² + … + v_n²) = √(Σ v_i²)

In ℝ² this is the Pythagorean theorem; in ℝ³ it is the diagonal of a box; in higher dimensions it is the natural generalization. The magnitude is always non-negative, and is zero only for the zero vector.

Normalization rescales a vector to unit length. If v ≠ 0, the unit vector in the same direction is v̂ = v / ||v||, so that ||v̂|| = 1 while the direction is preserved. Normalization strips away “how long” and keeps only “which way,” which is exactly what you want for surface normals in graphics, gradient directions in optimization, and direction-of-arrival vectors in signal processing.

The Euclidean norm is the most common, but not the only one. The L¹ norm ||v||₁ = Σ |v_i| sums absolute values and is used in robust regression and compressed sensing. The L∞ norm ||v||_∞ = max_i |v_i| picks the largest absolute entry and is used in worst-case error analysis. All norms agree on the zero vector, but they disagree on which non-zero vectors count as “large,” and that choice has real consequences in optimization.

A numerical pitfall: computing √(x² + y²) directly can overflow for very large inputs or lose precision for very small ones. Standard libraries provide hypot(x, y), which returns the same value while avoiding catastrophic cancellation and intermediate overflow. Use numpy.linalg.norm or math.hypot in production code.

벡터의 크기는 ||v|| = √(Σv_i²)로 정의되며, 정규화 v̂ = v/||v||는 방향만 남기고 길이를 1로 맞춥니다. L¹·L∞ 같은 다른 노름도 자주 쓰이고, 실제 코드에서는 오버플로·언더플로를 피하기 위해 `hypot`·`numpy.linalg.norm` 같은 라이브러리 함수를 사용해야 합니다.

7. Projection & Decomposition

A projection answers: how much of vector a points along vector b? The scalar projection is the length of the shadow that a casts onto the line spanned by b:

comp_b(a) = (a · b) / ||b||

It is a single number — positive if a leans in the same direction as b, negative if it leans the opposite way, zero if a is perpendicular to b. The vector projection turns that length into an actual vector pointing along b:

proj_b(a) = ((a · b) / (b · b)) · b

The denominator is b · b = ||b||², which is the correct rescaling so that when a equals b, the projection is b itself. If b is already a unit vector b̂, the formula simplifies to proj_b̂(a) = (a · b̂) · b̂.

The vector projection produces an orthogonal decomposition. Any vector a splits into a piece parallel to b and a piece perpendicular to b:

a = proj_b(a) + perp_b(a), where perp_b(a) = a − proj_b(a)

By construction perp_b(a) · b = 0. This is the geometric core of countless algorithms: separating “signal aligned with the model” from “residual error,” computing reflections (which flip the perpendicular component), and snapping a point onto the closest line or plane.

Projection is also the engine behind the Gram-Schmidt process, which turns linearly independent vectors into an orthonormal basis by repeatedly subtracting projections — keep u₁ = v₁, then u₂ = v₂ − proj_u1(v₂), then u₃ = v₃ − proj_u1(v₃) − proj_u2(v₃), and so on, normalizing each u_i at the end. This underlies QR decomposition, least-squares regression, and orthonormal frames in graphics.

투영은 한 벡터가 다른 벡터 방향으로 얼마나 가리키는지를 측정합니다. 스칼라 투영 (a·b)/||b||는 그림자의 길이, 벡터 투영 ((a·b)/(b·b))b는 그 그림자에 해당하는 실제 벡터이며, a = proj + perp의 분해는 최소제곱·반사·그람-슈미트 정규직교화의 기반이 됩니다.

8. Angle Between Vectors

Solving a · b = ||a|| · ||b|| · cos θ for cos θ gives the most useful formula in vector geometry:

cos θ = (a · b) / (||a|| · ||b||)

The right-hand side is always between −1 and +1 (Cauchy-Schwarz inequality), so it is always a valid cosine. To recover θ itself, take θ = arccos((a · b) / (||a|| ||b||)). The result lies in [0, π], matching the convention that “the angle between two vectors” is the smaller of the two possible directed angles.

A key consequence: two non-zero vectors are orthogonal (perpendicular) if and only if their dot product is zero. No angle or square root is needed — orthogonality is a clean algebraic test that powers collision response, projection onto subspaces, and rejection of redundant features in machine learning.

Worked example. Let a = (1, 2, 2) and b = (2, 1, −2). The dot product is 1·2 + 2·1 + 2·(−2) = 2 + 2 − 4 = 0, so the vectors are orthogonal regardless of their magnitudes; cos θ = 0 and θ = π/2 exactly.

In high dimensions the angle itself is often less meaningful than its cosine — the cosine similarity cos θ used as a similarity score. This dominates information retrieval (ranking TF-IDF document vectors) and modern ML (semantic search and RAG over sentence embeddings); ignoring magnitude is exactly the right behavior when document length or word frequency should not skew the comparison.

두 벡터 사이의 각도는 cos θ = (a·b) / (||a||·||b||)로 구하며, 내적이 0이면 두 벡터는 수직입니다. 머신러닝과 정보검색에서는 각도 자체보다도 cos θ 값(코사인 유사도)이 더 자주 쓰이며, 길이를 무시하고 방향 일치 정도만 비교하기에 의미·문서 유사도 측정에 적합합니다.

9. Common Mistakes & Pitfalls

• Confusing dot and cross products. The dot product returns a scalar; the cross product returns a vector. Mixing them up changes the shape of the result and breaks every downstream computation.
• Assuming every vector space is 3-dimensional. The cross product is meaningful only in ℝ³ (practically), but the dot product, magnitude, projection, and angle all work in any ℝⁿ. A 768-entry word embedding is just as much a vector as an arrow on a whiteboard.
• Mixing row and column vectors. Treating a 1×n row and n×1 column as interchangeable eventually produces a dimension-mismatch error — or, worse, a silently wrong answer when broadcasting hides it. Pick one convention (columns in most modern texts) and stick to it.
• Forgetting that cross product is anticommutative. a × b = −b × a. In graphics this flips a surface normal and inverts lighting; in physics it inverts torque direction.
• Normalizing the zero vector. Computing 0 / ||0|| is division by zero. Robust normalization code must check whether ||v|| is below a small threshold and return 0 or an error rather than NaN. This bug is especially common in physics engines where forces can briefly cancel.
• Equating “magnitude zero” with “no information.” A small-magnitude vector may still carry an important direction; a near-zero gradient still indicates which way to nudge parameters.
• Using the naive √(x² + y²) in numerical code. Large inputs can overflow before the square root completes. Always prefer the library hypot or norm.

가장 흔한 실수는 내적(스칼라)과 외적(벡터)의 결과 차이를 혼동하는 것, 외적이 3차원 전용임을 잊는 것, 행/열 벡터 혼용으로 인한 차원 오류, a × b = −b × a라는 반교환성 누락, 그리고 영벡터를 정규화하려다 NaN을 만드는 경우입니다.

10. Applications & Related Concepts

Vectors appear in nearly every quantitative discipline:

• Computer graphics. Every 3D model carries surface normals built as cross products of edge vectors; lighting reduces to dot products of normals with light directions; camera setups use orthonormal frames built by Gram-Schmidt. Without vectors, there is no rendering.
• Physics & engineering. Force, velocity, acceleration, momentum, and torque are all vectors. Statics sums force vectors to check equilibrium; dynamics integrates acceleration over time; torque is a literal cross product τ = r × F. Vector calculus extends the same ideas to fields that vary across space.
• Machine learning & NLP. Modern language models represent every word, sentence, and document as a high-dimensional embedding vector, and cosine similarity between embeddings powers semantic search, RAG, and recommendation systems. The attention mechanism in a Transformer is built from dot products of query and key vectors.
• Data science. Every row of a data table is a feature vector in ℝⁿ. Clustering (k-means, DBSCAN) measures distances between feature vectors; PCA finds directions of maximum variance; classifiers fit decision boundaries in vector space.
• Robotics & signal processing. A robot’s state and control input are vectors updated by linear feedback. A sampled audio clip is a vector in ℝⁿ; filters, correlations, and Fourier transforms are linear-algebra operations on it.

Vectors are the columns of matrices; for the geometry of angles see /docs/geo/; for high-dimensional vector statistics see /docs/stat/. Probability also enters whenever vector entries are random, as in noise models and Gaussian embeddings (/docs/prob/).

Practice → C:Vector

벡터는 컴퓨터 그래픽스(법선·조명), 물리·공학(힘·속도·토크), 머신러닝(임베딩과 코사인 유사도), 데이터 과학(특성 벡터·PCA), 로보틱스, 신호처리 등 거의 모든 정량적 분야에서 핵심이 됩니다. 다음 단계는 행렬(벡터의 열로 묶인 객체), 기하(각·길이·투영의 토대), 통계(공분산·PCA)이며, 실전 연습은 C:Vector에서 가능합니다.

11. Explore Each Vector Operation in Depth

Each core vector operation has its own focused, worked-example guide. Use these as deep dives whenever you need the formula, a step-by-step example, and the key properties for one specific operation:

• Vector Addition — the component rule and the tip-to-tail and parallelogram pictures.
• Vector Subtraction — a − b = a + (−b) and why the result points from b to a.
• Scalar Multiplication — scaling a vector by a number and when it flips direction.
• Dot Product — a · b = Σ aᵢbᵢ, the cos θ link, and the orthogonality test.
• Cross Product — the 3D perpendicular vector, the right-hand rule, and parallelogram area.
• Magnitude — the length formula |a| = √(Σ aᵢ²) and its link to the dot product.
• Unit Vector — normalizing with â = a / |a| to keep direction at length 1.
• Angle Between Vectors — cos θ = (a · b)/(|a| |b|) and reading the sign.
• Projection — the shadow of one vector on another and orthogonal decomposition.

각 핵심 연산마다 공식·단계별 예제·주요 성질을 담은 심화 가이드가 있습니다: 덧셈, 뺄셈, 스칼라배, 내적, 외적, 크기, 단위벡터, 벡터 사이 각, 투영.