假设你有非负的、大的权重值和小的激活值A(可能是sigmoid (z)的情况)。 Consider you have non-negative and large weights and small activations A(as can be the case for sigmoid (z)). 如果我们初始化较大的权重,激活将会很大,导致零斜率(在sigmoid 和tanh激活函数的情况下)。 If we initialize large weights, the activation will be large, resulting in zero slope(in case of sigmoid and tanh activation function). 因此,如果z非常大,exp(-z)将接近0,因此Sigmoid 的输出将是1。 So, if z is very large, exp(-z) will be close to 0, and therefore the output of the sigmoid will be 1. 大多数神经网络架构都需要标准化数据,因为tanh和sigmoid 等大多数神经元的激活函数都定义在[-1,1]或[0,1]区间内。 Because most common activation functions of the network's neurons such as tanh or sigmoid are defined on the[-1, 1] or[0, 1] interval respectively. 假设我们使用sigmoid 神经元,所有激活值都是正数。 Suppose we're using sigmoid neurons, so all activations in our network are positive.
示例包括对数几率(sigmoid )联系和对数联系。 Examples include the logit(sigmoid ) link and the log link. 显然,这样的神经元和sigmoid 和tanh都不一样。 Obviously such neurons are quite different from both sigmoid and tanh neurons. Sigmoid 激活函数创建一个在0和1之间的输出。Sigmoid activation function creates an output with values between 0 and 1.直接构建sigmoid 层需要使用3个layeroperations而不是一个. Directly composing sigmoid layers requires three layers of operation, instead of one. 在平常实践中,ReLU比Sigmoid 或者tanh函数表现的更好。 In general practice as well, ReLU has found to be performing better than sigmoid or tanh functions. 在早期,Sigmoid 函数和tanh函数是人们经常使用的激活函数。 Earlier Sigmoid and Tanh were the most widely used activation function. 主要的激活函数有5种,date,step,sigmoid ,tanh和ReLU。 There have been 5 major activation functions tried to date, step, sigmoid , tanh, and ReLU. 每个单位的条件概率分布由其接收的输入的logisticsigmoid 函数给出. The conditional probability distribution of each unit is given by the logistic sigmoid activation function of the input it receives.我们选择Sigmoid 还是Tanh是根据问题中对梯度的要求而定的。 Our choice of using sigmoid or tanh would basically depend on the requirement of gradient in the problem statement. The gradient of the Sigmoid curve can be found by taking the derivative. The next test is to check the performance of the sigmoid operation. 到目前为止,我们已经使用了Sigmoid 激活函数:. So far, we have used the sigmoid activation function. 函数sigmoid _kernel计算两个向量之间的sigmoid kernel。 The function sigmoid _kernel computes the sigmoid kernel between two vectors. 因此Logistic回归也使用Sigmoid 函数来输出概率:. So logistic regression also uses a sigmoid function to output probabilities. The next test is to check the performance of the sigmoid operation.
Display more examples
Results: 151 ,
Time: 0.0221