Convex vs. Non-Convex Functions: Why it Matters in Optimization …
Apr 18, 2023 · When working with non-convex functions, it is important to use optimization algorithms that can help us avoid getting stuck in local minima. For example, gradient descent with random...
[1712.07897] Non-convex Optimization for Machine Learning
Dec 21, 2017 · The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non …
Research direction 1: Understanding statistical + optimization landscape of important nonconvex problems E.g., SOSPs = global optima in matrix factorization problems
Global minimum Local minima Local randomized coordinate descent Strategy 1: local optimization of the non-convex function All convex functions rates apply.
Non-convex optimization is critical to several ML problems with applications in deep learning, recommendation systems (matrix completion), dimensionality reduction (PCA, sparse-PCA), robust …
The function is truly non-convex locally (i.e., r2f(x) has at least one very negative eigenvalue): do Hessian descent (which works better/converges very fast when the function is very non-convex).
Optimization viewpoint If we factorize Z = XX> with X ∈ Rn×r, then it leads to a nonconvex problem: minimizeX∈Rn×r 1 f(X) = kXX> − Mk2
Non-Convex Optimization: A Review - IEEE Xplore
This paper focuses upon Non-Convex Optimization Algorithms such as SGD, EM Algorithm, Alternating Minimization and its potential applications in the real world such as Low-Rank Matrix Recovery, …
Non-convex Optimization - IID Group
While efficient algorithms are known for a few instances of non-convex problems, it remains a central challenge to discover general conditions under which a non-convex problem admits an efficient solution.
Review Non-convex Optimization Method for Machine Learning
Oct 2, 2024 · This paper examines the key methods and applications of non-convex optimization in machine learning, exploring how it can lower computation costs while enhancing model performance.