Multi-Objective Optimization Concepts

9. Multi-Objective Optimization Concepts#

Note

This page introduces the fundamental concepts of multi-objective optimization and how they apply to materials design problems.

9.1. What is Multi-Objective Optimization?#

Multi-objective optimization (MOO) deals with problems where we need to optimize multiple, often conflicting objectives simultaneously. Unlike single-objective optimization that seeks one optimal solution, MOO typically results in a set of trade-off solutions called the Pareto front.

Real-World Example

Consider designing a new alloy:

Objective 1: Maximize strength (higher is better)
Objective 2: Maximize ductility (higher is better)
Objective 3: Minimize cost (lower is better)

These objectives often conflict: stronger alloys may be more brittle (less ductile) and more expensive.

9.2. Key Concepts#

9.2.1. Pareto Dominance#

A solution dominates another if it’s better in at least one objective and no worse in all others.

# Example: Two alloy compositions
Alloy_A = [Strength=250, Ductility=15, Cost=100]
Alloy_B = [Strength=240, Ductility=15, Cost=100]

# Alloy A dominates Alloy B because:
# - Strength: 250 > 240 (better)
# - Ductility: 15 = 15 (equal)
# - Cost: 100 = 100 (equal)

9.2.2. Pareto Front#

The Pareto front (or Pareto frontier) is the set of all non-dominated solutions. These represent the best possible trade-offs between objectives.

Strength vs. Ductility Trade-off:

Ductility
    ^
    |     * (Pareto optimal)
    |   *   * (Pareto optimal)
    | *       * (Pareto optimal)
    |           * (Pareto optimal)
    |             *
    +----------------> Strength
    
Points on the curve are Pareto optimal.
Points below the curve are dominated.

9.2.3. Pareto Optimality#

A solution is Pareto optimal if no other solution dominates it. The goal of multi-objective optimization is to find the Pareto front.

9.3. Mathematical Formulation#

9.3.1. General Multi-Objective Problem#

minimize/maximize: f(x) = [f₁(x), f₂(x), ..., fₘ(x)]
subject to: g(x) ≤ 0
           h(x) = 0
           x ∈ X

Where:

f(x): Vector of m objective functions
g(x): Inequality constraints
h(x): Equality constraints
X: Decision variable space

9.3.2. Materials Design Example#

For alloy optimization:

maximize: f₁(x) = Strength(Cu, Mg, Si, T_aging)
maximize: f₂(x) = Ductility(Cu, Mg, Si, T_aging)
minimize: f₃(x) = Cost(Cu, Mg, Si, T_aging)

subject to: Cu + Mg + Si ≤ 10%  (composition constraint)
           150 ≤ T_aging ≤ 200   (temperature constraint)
           Cu, Mg, Si ≥ 0        (non-negativity)

9.4. Multi-Objective vs. Single-Objective#

Table 9.1 Comparison#
Aspect	Single-Objective	Multi-Objective
Solution	One optimal point	Set of trade-off solutions (Pareto front)
Decision Making	Automatic	Requires preference information
Complexity	Lower	Higher (curse of dimensionality)
Visualization	Easy (1D or 2D)	Difficult (high-dimensional)
Algorithms	Well-established	More complex, newer field

9.5. Challenges in Multi-Objective Optimization#

9.5.1. 1. Conflicting Objectives#

Most real-world problems involve trade-offs:

# Materials examples
conflicts = {
    "Strength vs. Ductility": "Stronger materials are often more brittle",
    "Performance vs. Cost": "Better performance usually costs more",
    "Conductivity vs. Stability": "High conductivity may reduce thermal stability",
    "Hardness vs. Toughness": "Harder materials may be less tough"
}

9.5.2. 2. Curse of Dimensionality#

As the number of objectives increases, the problem becomes exponentially more complex:

2-3 objectives: Manageable, good visualization
4-6 objectives: Challenging, limited visualization
>6 objectives: Very difficult, many-objective optimization

9.5.3. 3. Solution Selection#

The Pareto front provides multiple solutions, but we often need to select one:

A priori: Define preferences before optimization
A posteriori: Choose from Pareto front after optimization
Interactive: Iteratively refine preferences during optimization

9.6. Multi-Objective Bayesian Optimization (MOBO)#

MOBO extends Bayesian optimization to handle multiple objectives by:

Surrogate Models: Build separate models for each objective
Multi-Objective Acquisition: Use acquisition functions that consider all objectives
Pareto Front Approximation: Iteratively improve the Pareto front estimate

9.6.1. Key Advantages#

Why MOBO for Materials?

Expensive Experiments: Materials testing is costly and time-consuming
Multiple Properties: Materials have many important properties
Trade-off Understanding: Need to understand relationships between properties
Efficient Exploration: Find Pareto front with minimal experiments

9.7. MOBO Acquisition Functions#

9.7.1. Expected Hypervolume Improvement (EHVI)#

EHVI maximizes the expected improvement in hypervolume (volume of dominated objective space):

EHVI(x) = E[HV(F ∪ {f(x)}) - HV(F)]

Where:

F: Current Pareto front approximation
HV: Hypervolume indicator
f(x): Objective vector at point x

Best for: 2-4 objectives, balanced exploration

9.7.2. Probability of Improvement (PI)#

Multi-objective PI considers improvement probability for each objective:

PI(x) = P(f(x) dominates at least one point in F)

Best for: Conservative optimization, exploitation-focused

9.7.3. Upper Confidence Bound (UCB)#

Multi-objective UCB balances mean prediction and uncertainty:

UCB(x) = μ(x) + β·σ(x)

Applied to each objective separately or combined.

Best for: Noisy objectives, exploration-focused

9.8. Hypervolume Indicator#

The hypervolume is a key quality indicator for Pareto fronts:

HV(F) = Volume of space dominated by F

9.8.1. Properties#

Monotonic: Larger hypervolume = better Pareto front
Pareto Compliant: Respects Pareto dominance
Reference Point Dependent: Requires a reference point

9.8.2. Calculation Example#

For 2D objectives (maximize both):

# Pareto front points
pareto_points = [(3, 1), (2, 2), (1, 3)]
reference_point = (0, 0)

# Hypervolume = sum of dominated rectangles
# Rectangle 1: (3-0) × (1-0) = 3
# Rectangle 2: (2-0) × (2-1) = 2  
# Rectangle 3: (1-0) × (3-2) = 1
# Total HV = 3 + 2 + 1 = 6

9.9. Practical Considerations#

9.9.1. Objective Scaling#

Different objectives may have different scales:

# Before scaling
objectives = {
    "Strength": [200, 300, 250],      # MPa
    "Ductility": [10, 20, 15],       # %
    "Cost": [50, 100, 75]            # $/kg
}

# After normalization (0-1 scale)
normalized = {
    "Strength": [0.0, 1.0, 0.5],
    "Ductility": [0.0, 1.0, 0.5], 
    "Cost": [0.0, 1.0, 0.5]
}

9.9.2. Reference Point Selection#

For hypervolume calculation, choose reference points carefully:

# Good reference point (slightly worse than worst known values)
reference_point = [min_strength - 0.1*range_strength,
                  min_ductility - 0.1*range_ductility,
                  max_cost + 0.1*range_cost]  # Note: max for minimization

9.9.3. Objective Transformation#

Convert minimization to maximization:

# Original objectives
strength = 250      # maximize
ductility = 15      # maximize  
cost = 100          # minimize

# Transformed for maximization
objectives = [strength, ductility, -cost]  # Negate cost

9.10. Materials Design Applications#

9.10.1. Alloy Composition Optimization#

# Objectives for structural alloys
objectives = [
    "Yield Strength",     # Maximize
    "Ultimate Strength",  # Maximize
    "Elongation",        # Maximize
    "Fatigue Life",      # Maximize
    "Cost",              # Minimize
    "Density"            # Minimize (for aerospace)
]

9.10.2. Processing Parameter Optimization#

# Heat treatment optimization
objectives = [
    "Hardness",          # Maximize
    "Toughness",         # Maximize
    "Residual Stress",   # Minimize
    "Energy Cost",       # Minimize
    "Processing Time"    # Minimize
]

9.10.3. Multi-Functional Materials#

# Electronic materials
objectives = [
    "Electrical Conductivity",  # Maximize
    "Thermal Conductivity",     # Maximize/Minimize (depends on application)
    "Mechanical Strength",      # Maximize
    "Corrosion Resistance",     # Maximize
    "Manufacturing Cost"        # Minimize
]

9.11. Visualization Techniques#

9.11.1. 2D Pareto Front#

import matplotlib.pyplot as plt

# Plot Pareto front
plt.scatter(strength_values, ductility_values, c='red', label='Pareto Front')
plt.xlabel('Strength (MPa)')
plt.ylabel('Ductility (%)')
plt.title('Strength vs. Ductility Trade-off')

9.11.2. 3D Pareto Front#

from mpl_toolkits.mplot3d import Axes3D

fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(strength, ductility, cost, c='red')
ax.set_xlabel('Strength')
ax.set_ylabel('Ductility') 
ax.set_zlabel('Cost')

9.11.3. Parallel Coordinates#

For >3 objectives, use parallel coordinate plots:

import pandas as pd
from pandas.plotting import parallel_coordinates

# Create DataFrame with objectives
df = pd.DataFrame({
    'Strength': strength_values,
    'Ductility': ductility_values,
    'Cost': cost_values,
    'Corrosion': corrosion_values,
    'Type': 'Pareto'
})

parallel_coordinates(df, 'Type')

9.12. Decision Making#

9.12.1. Preference-Based Selection#

After finding the Pareto front, select solutions based on preferences:

# Weight-based selection
weights = [0.4, 0.3, 0.3]  # [strength, ductility, cost]

# Calculate weighted sum for each Pareto solution
scores = []
for solution in pareto_front:
    score = sum(w * obj for w, obj in zip(weights, solution))
    scores.append(score)

# Select solution with highest score
best_index = np.argmax(scores)
selected_solution = pareto_front[best_index]

9.12.2. Knee Point Selection#

The “knee point” represents the best compromise:

# Find knee point (maximum distance from line connecting extremes)
def find_knee_point(pareto_front):
    # Implementation depends on specific algorithm
    # Common approaches: maximum perpendicular distance, angle-based
    pass

9.13. Next Steps#

Now that you understand multi-objective concepts:

Learn MOBO algorithms: MOBO Algorithms in MultiBgolearn
Try MultiBgolearn: MultiBgolearn: Multi-Objective Bayesian Global Optimization
Practice with examples: Multi-Objective Optimization Examples
Explore Pareto optimization: Pareto Optimization and Trade-off Analysis