演講公告
新聞標題： ( 2020-06-09 )
演講主題：Un-rectifying Non-linear networks
主講人：黃文良研究員(中央研究院資訊科學研究所)
演講日期：2020年6月16日(星期二) 14:00 –15:00
演講地點：(光復校區) 科學一館223室
茶會時間：當天下午13:30 (科學一館205室)
摘要內容：
Abstract. I invent the novel "un-rectifying" technique for analysis of deep neural network (DNN).
Deep feedforward neural networks with piecewise linear activations can be viewed as piecewise affine functions, affine linear on polytopes partitioning the input space. We thus consider networks with rectified linear units and max-pooling operations from a signal representation perspective. In this view, such representations mark the significant transition from using a single linear representation to utilizing a large collection of affine linear representations tailored to particular regions of the signal space. However, the expression power of a network cannot be fully leveraged in signal processing without explicit expressions of the affine linear operators, their domains, ranges, and composition from the weight and bias parameters of the network. This article addresses the problem and provides a precise description of the individual affine representations and corresponding domain regions that the neural network associates to each signal of the input space. In particular, we describe weighted atomic decompositions of the representations and, based on estimating their Lipschitz regularity, draw a connection between sparse or compressible weight distributions and the stability in representation and learning, independent of the network depth. Such analysis may facilitate understanding networks and promote further theoretical insight from both the signal processing and machine learning communities.相關檔案：talk_1090616.pdf
go back